Does Typological Distance Affect Cross-lingual Transfer?

Pengfei Ren's BabyLM 2026 paper finds that coarse near/far typological distance between a first language (L1) and second language (L2) does not produce a uniform, target-independent advantage for the near L1 when GPT-2 Small models are trained from scratch on English, Dutch, or Chinese with a fixed Qwen2.5 tokenizer and then continued on the L2 target. Across five seeds, NL→EN and ZH→EN showed nearly identical Transfer Increment for English, and for Dutch the near-L1 condition's larger Transfer Increment came mostly from zero-shot starting scores, with terminal scores differing by only 0.9 percentage points. Headroom-normalized Transfer Increment gave the same target-dependent picture, the paper reports in Proceedings of the Second BabyLM Workshop, pages 65–74, Budapest, Hungary.

Abstract The Contrastive-Analysis Hypothesis CAH proposes that similarities between a learner’s first language L1 and second language L2 can facilitate L2 learning, while differences can make it more difficult. We ask a related computational question: under a limited, developmentally motivated data budget, does a coarse near/far relation between L1 and L2 predict grammatical performance when a language model is trained first on L1 and then on L2? We train GPT-2 Small models from scratch on English, Dutch, or Chinese with a fixed Qwen2.5 tokenizer, and continue training them on the corresponding L2 targets. Across five seeds, we compare zero-shot starting scores, terminal L2 scores, Transfer Increment TI; terminal minus zero-shot , and Transfer Benefit TB; terminal minus a same-budget target-only baseline . For English, NL→EN and ZH→EN have nearly identical TI. For Dutch, the near-L1 condition has a larger TI, but most of this difference comes from the zero-shot starting scores, while the terminal scores differ by only 0.9 percentage points. Headroom-normalized TI gives the same target-dependent picture. Across the tested pairs, the near/far contrast therefore does not produce a uniform, target-independent advantage for the near L1. - Anthology ID: - 2026.babylm-main.6 - Volume: - Proceedings of the Second BabyLM Workshop https://aclanthology.org/volumes/2026.babylm-main/ - Month: - October - Year: - 2026 - Address: - Budapest, Hungary - Editors: - Leshem Choshen https://aclanthology.org/people/leshem-choshen/ , Ryan Cotterell https://aclanthology.org/people/ryan-cotterell/ , Mustafa Omer Gul https://aclanthology.org/people/mustafa-omer-gul/ , Jaap Jumelet https://aclanthology.org/people/jaap-jumelet/unverified/ , Tal Linzen https://aclanthology.org/people/tal-linzen/ , Aaron Mueller https://aclanthology.org/people/aaron-mueller/ , Suchir Salhan https://aclanthology.org/people/suchir-salhan/ , Raj Sanjay Shah https://aclanthology.org/people/raj-sanjay-shah/ , Alex Warstadt https://aclanthology.org/people/alex-warstadt/ , Ethan Gotlieb Wilcox https://aclanthology.org/people/ethan-wilcox/ - Venues: - BabyLM https://aclanthology.org/venues/babylm/ | WS https://aclanthology.org/venues/ws/ - SIG: - Publisher: - Association for Computational Linguistics - Note: - Pages: - 65–74 - Language: - URL: - https://aclanthology.org/2026.babylm-main.6/ https://aclanthology.org/2026.babylm-main.6/ - DOI: - Cite ACL : - Pengfei Ren. 2026. Does Typological Distance Affect Cross-lingual Transfer? https://aclanthology.org/2026.babylm-main.6/ . In Proceedings of the Second BabyLM Workshop , pages 65–74, Budapest, Hungary. Association for Computational Linguistics. - Cite Informal : - Does Typological Distance Affect Cross-lingual Transfer? https://aclanthology.org/2026.babylm-main.6/ Ren, BabyLM 2026 - PDF: - https://aclanthology.org/2026.babylm-main.6.pdf https://aclanthology.org/2026.babylm-main.6.pdf