04:00
2026-08-10
arxiv.org
machine-learning
EntropyMoE: Entropy-Aware Sparse Expert Routing for Tokenizer-Free LLMs
Researchers introduced EntropyMoE, a Mixture-of-Experts architecture for byte-level large language models that routes tokens based on patch entropy, achieving the lowest held-out bits-per-byte among m…