{"slug": "edward-nmt-elastic-dimensions-with-adaptive-routing-and-dual-consistency-for", "title": "EDWARD-NMT: Elastic Dimensions With Adaptive Routing and Dual-consistency for Neural Machine Translation", "summary": "Researchers posted arXiv paper 2610.32115v1 introducing EDWARD-NMT, a sparse Mixture-of-Experts encoder-decoder architecture for neural machine translation that exposes a family of deployable sub-networks from a single trained checkpoint. EDWARD-NMT combines Matryoshka-style elastic dimensions, an adaptive sigmoid router with a learned task-conditioned threshold that lets each token set its own active expert count, and a dual-consistency objective; experiments on PhoMT (En-Vi), WMT-14 (En-De), multilingual OPUS-16 and a multi-domain De-En benchmark show the full-capacity model matches or exceeds strong MoE baselines with improved parameter efficiency, and extracted sub-networks consistently outperform comparably sized dense Transformers without retraining.", "body_md": "arXiv:2610.32115v1 Announce Type: new \nAbstract: Sparse Mixture-of-Experts (MoE) is a powerful paradigm for scaling Neural Machine Translation (NMT) and managing language heterogeneity. However, practical limitations hinder its deployment on resource-constrained devices: the entire expert pool must remain in memory, and the per-token compute budget is statically fixed during training by rigid top-k routing. We introduce EDWARD-NMT (Elastic Dimensions With Adaptive Routing and Dual-consistency for NMT), a novel MoE encoder-decoder architecture designed for a train once, deploy anywhere paradigm that exposes a family of deployable sub-networks at inference time. EDWARD-NMT integrates three key innovations: (i) Matryoshka-style elastic dimensions to flexibly reduce the memory footprint; (ii) an adaptive sigmoid router with a learned, task-conditioned threshold, allowing each token to dynamically determine its active expert count; and (iii) a dual-consistency objective to maximize the performance of the extracted sub-networks. Extensive experiments on PhoMT (En-Vi), WMT-14 (En-De), multilingual OPUS-16, and a multi-domain De-En benchmark validate our approach. The full-capacity architecture matches or exceeds strong MoE baselines with improved parameter efficiency. Furthermore, the sub-networks seamlessly extracted from a single checkpoint consistently outperform comparably sized dense Transformers without any retraining.", "url": "https://wpnews.pro/news/edward-nmt-elastic-dimensions-with-adaptive-routing-and-dual-consistency-for", "canonical_source": "https://www.machinebrief.com/news/edward-nmt-elastic-dimensions-with-adaptive-routing-and-dual-elgr", "published_at": "2026-10-26 04:00:00+00:00", "updated_at": "2026-10-26 04:41:54.494103+00:00", "lang": "en", "topics": ["machine-learning", "natural-language-processing", "ai-research", "large-language-models"], "entities": ["EDWARD-NMT", "arXiv", "PhoMT", "WMT-14", "OPUS-16"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/edward-nmt-elastic-dimensions-with-adaptive-routing-and-dual-consistency-for", "markdown": "https://wpnews.pro/news/edward-nmt-elastic-dimensions-with-adaptive-routing-and-dual-consistency-for.md", "text": "https://wpnews.pro/news/edward-nmt-elastic-dimensions-with-adaptive-routing-and-dual-consistency-for.txt", "jsonld": "https://wpnews.pro/news/edward-nmt-elastic-dimensions-with-adaptive-routing-and-dual-consistency-for.jsonld"}}