04:00
2026-09-21
arxiv.org
machine-learning
Attention-Aware Routing: Coupling Routing and Attention in MoEs
Attention-Aware Routing (AAR), a method that augments a Mixture-of-Experts router with temporal and spectral features from a sliding window of attention weights, improved GSM8K accuracy by 3.37 percen…