mBART-50

Researchers propose a corpus-driven vocabulary pruning framework for multilingual neural machine translation, reducing vocabulary from over 128,000 to approximately 10,000 tokens and achieving a 60% m…

Researchers have developed BHARATI, a set of SentencePiece BPE tokenizers for classical Indian languages, achieving a 90% reduction in sequence length compared to GPT-2 on a held-out test set of 490 I…