cd /news/machine-learning/efficient-multilingual-neural-machin… · home topics machine-learning article
[ARTICLE · art-87264] src=machinebrief.com ↗ pub= topic=machine-learning verified=true sentiment=↑ positive

Efficient Multilingual Neural Machine Translation via Corpus-Driven Vocabulary Pruning: An English-Arabic Case Study

Researchers propose a corpus-driven vocabulary pruning framework for multilingual neural machine translation, reducing vocabulary from over 128,000 to approximately 10,000 tokens and achieving a 60% memory saving without performance loss. Evaluated on English-Arabic with M2M100, NLLB-200, and mBART-50, the pruned M2M100 model achieves a BLEU score of 42.04 and a COMET score of 0.8730, outperforming the OPUS-MTen-ar bilingual model on COMET (0.7911).

read1 min views1 publishedAug 5, 2026

arXiv:2608.03480v1 Announce Type: new Abstract: The adoption of large pre-trained multilingual models for neural machine translation (MNMT) faces a major challenge: excessive memory and computational consumption due to overly large vocabularies and embedding layers. Although existing compression methods like pruning, quantization and knowledge distillation reduce parameter redundancy, they mainly preserve the structure of the original vocabulary, thereby leaving a major source of inefficiency unresolved. We propose in this paper a general optimization framework that combines a vocabulary pruning method with a targeted fine-tuning protocol for MNMT models. We evaluate the proposed framework using three models (M2M100, NLLB-200, mBART-50) on the English-Arabic language pair. Our approach reduces the vocabulary size from over 128,000 to approximately 10,000 tokens, enabling a 60% memory saving without any loss in performance. Results show that optimized multilingual models can match or exceed the performance of dedicated bilingual baselines. In particular, the pruned and fine-tuned M2M100 model achieves a competitive BLEU score of 42.04 (against 44.59 for the OPUS-MTen- ar bilingual model) while it significantly outperforms it on the COMET metric (0.8730 vs 0.7911) revealing superior semantic adequacy and fluency.

── more in #machine-learning 4 stories · sorted by recency
── more on @m2m100 3 stories trending now
sponsored brought to you by zahid.host 4,200+ EU-deployed projects
reading about agents? ship yours in a single git push.

Run your AI side-project on zahid.host

EU-based hosting, git-push deploys, automatic HTTPS, no cold starts. Free tier with a custom domain — perfect for shipping the agent you just read about.

$git push zahid main
Live at https://your-agent.zahid.host
Get free account → Pricing
from €0/mo · no card required
LIVE [news/efficient-multilingu…] indexed:0 read:1min 2026-08-05 ·