{"slug": "multilingual-knowledge-transfer-under-data-constraints-via-lexical-interventions", "title": "Multilingual Knowledge Transfer under Data Constraints via Lexical Interventions", "summary": "Researchers from the University of Amsterdam and other institutions propose LINK, a data-level intervention method that improves cross-lingual knowledge transfer in multilingual language models by substituting random words in English pretraining data with their word-level translations using bilingual vocabularies, requiring no additional model training. Evaluated on eight languages across five model sizes, LINK achieves up to a 2x speedup in training to reach equivalent performance on downstream tasks in target languages.", "body_md": "[content type paper](/research/)published August 2026\n\nMultilingual Knowledge Transfer under Data Constraints via Lexical Interventions\n\nAuthorsAnastasiia Sedova, Natalie Schluter*, Skyler Seto*, Maartje ter Hoeve*\n\nMultilingual Knowledge Transfer under Data Constraints via Lexical Interventions\n\nAuthorsAnastasiia Sedova, Natalie Schluter*, Skyler Seto*, Maartje ter Hoeve*\n\nCross-lingual knowledge transfer is critical for building high-performing multilingual language models for languages with insufficient training data. When target language data is scarce, the knowledge required for many downstream tasks involving scientific reasoning, commonsense inference, and world knowledge must be acquired primarily from the high-resource language, making effective knowledge transfer essential. Existing methods for improving such cross-lingual knowledge transfer require large amounts of parallel data, translation systems, auxiliary models, or additional training stages that are largely unavailable for many languages. We propose LINK – a data-level intervention method that improves knowledge transfer during model pretraining through lexical substitutions in high-resource part of pretraining data using bilingual vocabularies. For a given replacement ratio, randomly selected words in a portion of the high-resource (English) training corpus are swapped with their word-level translations, requiring no additional model training and only a bilingual vocabulary, which can be obtained at near-zero cost for virtually any language. Evaluation on eight languages across five model sizes shows notable improvements on downstream tasks in the target language, with up to a 2x speedup in training to reach equivalent performance.\n\nCross-lingual Knowledge Transfer and Iterative Pseudo-labeling for Low-Resource Speech Recognition with Transducers\n\nJune 16, 2023[research area Speech and Natural Language Processing](/research/?domain=Speech%20and%20Natural%20Language%20Processing)\n\nVoice technology has become ubiquitous recently. However, the accuracy, and hence experience, in different languages varies significantly, which makes the technology not equally inclusive. The availability of data for different languages is one of the key factors affecting accuracy, especially in training of all-neural end-to-end automatic speech recognition systems.\n\nCross-lingual knowledge transfer and iterative pseudo-labeling are two…\n\nLanguages You Know Influence Those You Learn: Impact of Language Characteristics on Multi-Lingual Text-to-Text Transfer\n\nDecember 11, 2022[research area Speech and Natural Language Processing](/research/?domain=Speech%20and%20Natural%20Language%20Processing)[Workshop at NeurIPS](/research/?event=NeurIPS%20Workshop)\n\nMulti-lingual language models (LM), such as mBERT, XLM-R, mT5, mBART, have been remarkably successful in enabling natural language tasks in low-resource languages through cross-lingual transfer from high-resource ones. In this work, we try to better understand how such models, specifically mT5, transfer *any* linguistic and semantic knowledge across languages, even though no explicit cross-lingual signals are provided during pre-training. Rather,…", "url": "https://wpnews.pro/news/multilingual-knowledge-transfer-under-data-constraints-via-lexical-interventions", "canonical_source": "https://machinelearning.apple.com/research/multilingual-knowledge-transfer-lexical-interventions", "published_at": "2026-08-20 00:00:00+00:00", "updated_at": "2026-08-20 15:13:57.275657+00:00", "lang": "en", "topics": ["natural-language-processing", "large-language-models", "machine-learning"], "entities": ["University of Amsterdam", "LINK"], "alternates": {"html": "https://wpnews.pro/news/multilingual-knowledge-transfer-under-data-constraints-via-lexical-interventions", "markdown": "https://wpnews.pro/news/multilingual-knowledge-transfer-under-data-constraints-via-lexical-interventions.md", "text": "https://wpnews.pro/news/multilingual-knowledge-transfer-under-data-constraints-via-lexical-interventions.txt", "jsonld": "https://wpnews.pro/news/multilingual-knowledge-transfer-under-data-constraints-via-lexical-interventions.jsonld"}}