Efficient methods for building LLMs for low-resourced languages

Nalin Kumar's position paper, presented at the 1st Workshop for Young Researchers in Natural Language Generation (YNLG) in Hanoi, Vietnam, in October 2025, proposes two efficient strategies for building large language models (LLMs) for low-resourced languages: modular training, which tunes only non-embedding parameters after learning language-specific tokenizers and embeddings, and artificial language initialization, which uses structurally biased synthetic languages for faster, parameter-efficient pretraining. The paper outlines these findings and plans for future research and round-table discussions.

Abstract Large Language Models LLMs excel in many NLP tasks but remain biased toward high-resource languages. This position paper discusses the author’s current findings on efficient strategies for low-resource settings: i modular training, where only non-embedding parameters are tuned after learning language-specific tokenizers and embeddings, and ii artificial language initialization, which leverages structurally biased synthetic languages for faster, parameter-efficient pretraining. The paper also shares plans for future research and topics that the author would like to discuss during the round-table.- Anthology ID: - 2025.ynlg-main.5 - Volume: Proceedings of the 1st Workshop for Young Researchers in Natural Language Generation /volumes/2025.ynlg-main/ - Month: - October - Year: - 2025 - Address: - Hanoi, Vietnam - Editors: Alyssa Allen /people/alyssa-allen/unverified/ , Nils Feldhus /people/nils-feldhus/ , Rudali Huidrom /people/rudali-huidrom/unverified/ , Michela Lorandi /people/michela-lorandi/ , Adarsa Sivaprasad /people/adarsa-sivaprasad/ , Patrícia Schmidtová /people/patricia-schmidtova/ - Venue: YNLG /venues/ynlg/ - SIG: SIGGEN /sigs/siggen/ - Publisher: - Association for Computational Linguistics - Note: - Pages: - 21–23 - Language: - URL: https://aclanthology.org/2025.ynlg-main.5/ https://aclanthology.org/2025.ynlg-main.5/ - DOI: - Cite ACL : - Nalin Kumar. 2025. Efficient methods for building LLMs for low-resourced languages https://aclanthology.org/2025.ynlg-main.5/ . In Proceedings of the 1st Workshop for Young Researchers in Natural Language Generation , pages 21–23, Hanoi, Vietnam. Association for Computational Linguistics. - Cite Informal : Efficient methods for building LLMs for low-resourced languages https://aclanthology.org/2025.ynlg-main.5/ Kumar, YNLG 2025 - PDF: https://aclanthology.org/2025.ynlg-main.5.pdf https://aclanthology.org/2025.ynlg-main.5.pdf