When transformers learn "impossible" languages, what do they learn?
Researchers at arXiv trained GPT-2 models on 'impossible' languages and found that while grammatical sensitivity degrades gradually, generative production fails significantly, suggesting transmission …