{"slug": "ne-bert-a-multilingual-language-model-for-nine-northeast-indian-languages", "title": "NE-BERT: A Multilingual Language Model for Nine Northeast Indian Languages", "summary": "Researchers introduced NE-BERT, a multilingual encoder model trained on 8.3 million sentences across 9 Northeast Indian languages plus Hindi and English, which outperforms IndicBERT-V2 and MuRIL on all 9 languages with 15.97x and 7.64x lower average perplexity, respectively, and 1.50x better tokenization fertility than mBERT. The model, released under CC-BY-4.0, addresses vocabulary fragmentation in low-resource languages like Pnar and Kokborok through aggressive upsampling, and downstream part-of-speech tagging validates its practical utility.", "body_md": "arXiv:2608.18094v1 Announce Type: new\nAbstract: Large pretrained language models have demonstrated remarkable capabilities across diverse languages, yet critically underrepresented low-resource languages remain marginalized. We present NE-BERT, a domain-specific multilingual encoder model trained on approximately 8.3 million sentences spanning 9 Northeast Indian languages and 2 anchor languages (Hindi, English), a linguistically diverse region with minimal representation in existing multilingual models. By employing weighted data sampling and a custom SentencePiece Unigram tokenizer, NE-BERT outperforms IndicBERT-V2 and MuRIL across all 9 Northeast Indian languages, achieving 15.97X and 7.64X lower average perplexity respectively, with 1.50X better tokenization fertility than mBERT. We address critical vocabulary fragmentation issues in extremely low-resource languages such as Pnar (1,002 sentences) and Kokborok (2,463 sentences) through aggressive upsampling strategies. Downstream evaluation on part-of-speech tagging validates practical utility on three Northeast Indian languages. We release NE-BERT, test sets, and training corpus under CC-BY-4.0 to support NLP research and digital inclusion for Northeast Indian communities.", "url": "https://wpnews.pro/news/ne-bert-a-multilingual-language-model-for-nine-northeast-indian-languages", "canonical_source": "https://arxiv.org/abs/2608.18094", "published_at": "2026-08-20 04:00:00+00:00", "updated_at": "2026-08-20 04:12:50.110583+00:00", "lang": "en", "topics": ["large-language-models", "natural-language-processing", "artificial-intelligence"], "entities": ["NE-BERT", "IndicBERT-V2", "MuRIL", "mBERT", "SentencePiece", "Pnar", "Kokborok", "arXiv"], "alternates": {"html": "https://wpnews.pro/news/ne-bert-a-multilingual-language-model-for-nine-northeast-indian-languages", "markdown": "https://wpnews.pro/news/ne-bert-a-multilingual-language-model-for-nine-northeast-indian-languages.md", "text": "https://wpnews.pro/news/ne-bert-a-multilingual-language-model-for-nine-northeast-indian-languages.txt", "jsonld": "https://wpnews.pro/news/ne-bert-a-multilingual-language-model-for-nine-northeast-indian-languages.jsonld"}}