{"slug": "training-a-tiny-language-model-to-follow-prompts-across-6-low-resource-lanuages", "title": "Training a Tiny Language Model to Follow Prompts Across 6 Low Resource Lanuages", "summary": "BULaMU creator mwebazarick released a 47M-parameter language model fine-tuned to generate stories across six low-resource East African languages plus English, trained on a corpus expanded tenfold to 40 billion tokens. The expanded corpus adds Swahili, Lao, Amharic, Somali, Kinyarwanda and English to Luganda, with East African languages making up nearly 90% of the new training data. The fine-tuned model, BULaMU-3-42M-Story, is published on Hugging Face with a Streamlit demo, and the BULaMU family (20M, 47M and 110M parameters) is designed to run offline on a phone without a GPU or internet connection.", "body_md": "Hi Everybody,\n\nI am just writing to share some progress I have made on my project. I am the creator of BULaMU, a family of language models (20M, 47M, 110M) trained completely from scratch for a low resource language: Luganda. The models are small and compute-efficient enough to run offline on a phone without requiring a GPU or internet connection.\n\nOver the last few months, I increased the corpus ten fold (to 40 billion tokens) by including text from 5 other East African Languages (Swahili, Lao, Amharic, Somali, Kinyaranda) and English. East African Languages make up nearly 90% of the new training corpus.\n\nI trained a 47M parameter model on this corpus and then fine-tuned it to be able to generate responses (stories) between the different languages. I have attached a link below if you would like to try a demo of it. I also made the models available on Huggingface if you would like to experiment with them on your own.\n\nDemo: [https://tinyafricanstories.streamlit.app/](https://tinyafricanstories.streamlit.app/)\n\nFine-tuned (“Story-telling”) model: [mwebazarick/BULaMU-3-42M-Story · Hugging Face](https://huggingface.co/mwebazarick/BULaMU-3-42M-Story)", "url": "https://wpnews.pro/news/training-a-tiny-language-model-to-follow-prompts-across-6-low-resource-lanuages", "canonical_source": "https://discuss.huggingface.co/t/training-a-tiny-language-model-to-follow-prompts-across-6-low-resource-lanuages/191810#post_1", "published_at": "2026-11-13 17:53:36+00:00", "updated_at": "2026-11-13 18:21:14.668470+00:00", "lang": "en", "topics": ["large-language-models", "natural-language-processing", "ai-research", "ai-tools"], "entities": ["BULaMU", "mwebazarick", "Hugging Face", "Luganda", "Swahili", "Amharic", "Somali", "Kinyarwanda"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/training-a-tiny-language-model-to-follow-prompts-across-6-low-resource-lanuages", "markdown": "https://wpnews.pro/news/training-a-tiny-language-model-to-follow-prompts-across-6-low-resource-lanuages.md", "text": "https://wpnews.pro/news/training-a-tiny-language-model-to-follow-prompts-across-6-low-resource-lanuages.txt", "jsonld": "https://wpnews.pro/news/training-a-tiny-language-model-to-follow-prompts-across-6-low-resource-lanuages.jsonld"}}