cd /news/machine-learning/do-mixture-of-experts-routers-benefi… · home › topics › machine-learning › article
[ARTICLE · art-170322] src=arxiv.org ↗ pub= topic=machine-learning verified=true sentiment=· neutral

Do Mixture-of-Experts Routers Benefit from Ensembling?

A new arXiv paper (2611.17472v1) reports that ensembling the routers in mixture-of-experts models lowers validation loss as more routers are added, but the gain is offset by the computational cost of training the additional routers. The authors, who developed a performance-diversity decomposition for router ensembles, found that training routers to explicitly maximize expert diversity improved mean performance on three of four tasks (two image-classification and two tabular-regression), while explicitly maximizing router diversity had no effect, and combining router weights with the arithmetic mean beat the geometric mean. The results suggest routing benefits only mildly from ensembling.

by read1 min views4 publishedNov 16, 2026

arXiv:2611.17472v1 Announce Type: new Abstract: Ensembles of neural networks have repeatedly been shown to outperform single networks. Ensemble theory attributes this gain to the accuracy of the individual members and the diversity among them. Mixture-of-experts models already combine many experts, but a single router decides how the experts are weighted for each input. Motivated by ensemble theory, we apply ensembling to the routing step: an ensemble of routers, trained jointly with the experts, proposes the expert weights. We systematically evaluate this approach using underparameterized networks on two image-classification and two tabular-regression tasks. We analyse each experiment with a performance-diversity decomposition of the router ensemble, which, to our knowledge, has not previously been developed for routers. We report four findings. First, more routers lower the validation loss. Second, training routers to explicitly maximize expert diversity improves the mean performance on three of the four tasks. Third, explicitly maximizing router diversity has no effect on performance. Fourth, combining router weights with the arithmetic mean works better than combining them with the geometric mean. Routing benefits mildly from ensembling, but this benefit is offset by the computational cost of training additional routers.

── more in #machine-learning 4 stories · sorted by recency
── more on @arxiv 3 stories trending now
sponsored brought to you by zahid.host 4,200+ EU-deployed projects
reading about agents? ship yours in a single git push.

Run your AI side-project on zahid.host

EU-based hosting, git-push deploys, automatic HTTPS, no cold starts. Free tier with a custom domain — perfect for shipping the agent you just read about.

$git push zahid main
→ Live at https://your-agent.zahid.host ✓
Get free account → Pricing
from €0/mo · no card required
LIVE [news/do-mixture-of-expert…] indexed:0 read:1min 2026-11-16 · —