# Efficient methods for building LLMs for low-resourced languages

> Source: <https://aclanthology.org/2025.ynlg-main.5/>
> Published: 2026-08-20 00:00:00+00:00

##### Abstract

Large Language Models (LLMs) excel in many NLP tasks but remain biased toward high-resource languages. This position paper discusses the author’s current findings on efficient strategies for low-resource settings: (i) modular training, where only non-embedding parameters are tuned after learning language-specific tokenizers and embeddings, and (ii) artificial language initialization, which leverages structurally biased synthetic languages for faster, parameter-efficient pretraining. The paper also shares plans for future research and topics that the author would like to discuss during the round-table.- Anthology ID:
- 2025.ynlg-main.5
- Volume:
[Proceedings of the 1st Workshop for Young Researchers in Natural Language Generation](/volumes/2025.ynlg-main/)- Month:
- October
- Year:
- 2025
- Address:
- Hanoi, Vietnam
- Editors:
[Alyssa Allen](/people/alyssa-allen/unverified/),[Nils Feldhus](/people/nils-feldhus/),[Rudali Huidrom](/people/rudali-huidrom/unverified/),[Michela Lorandi](/people/michela-lorandi/),[Adarsa Sivaprasad](/people/adarsa-sivaprasad/),[Patrícia Schmidtová](/people/patricia-schmidtova/)- Venue:
[YNLG](/venues/ynlg/)- SIG:
[SIGGEN](/sigs/siggen/)- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 21–23
- Language:
- URL:
[https://aclanthology.org/2025.ynlg-main.5/](https://aclanthology.org/2025.ynlg-main.5/)- DOI:
- Cite (ACL):
- Nalin Kumar. 2025.
[Efficient methods for building LLMs for low-resourced languages](https://aclanthology.org/2025.ynlg-main.5/). In*Proceedings of the 1st Workshop for Young Researchers in Natural Language Generation*, pages 21–23, Hanoi, Vietnam. Association for Computational Linguistics. - Cite (Informal):
[Efficient methods for building LLMs for low-resourced languages](https://aclanthology.org/2025.ynlg-main.5/)(Kumar, YNLG 2025)- PDF:
[https://aclanthology.org/2025.ynlg-main.5.pdf](https://aclanthology.org/2025.ynlg-main.5.pdf)