{"slug": "draftexpert-expansion-aware-self-speculative-decoding-for-end-device-moe", "title": "DraftExpert: Expansion-Aware Self-Speculative Decoding for End-Device MoE Inference", "summary": "Researchers propose DraftExpert, an expansion-aware self-speculative decoding framework for expert-offloaded Mixture-of-Experts (MoE) inference on end devices, achieving 1.45x average decode throughput improvement, 84-87% draft acceptance, and 86-88% prefetch hit rates on DeepSeek-V2-Lite and Moonlight-16B-A3B across CPU-GPU and Flash-NPU offload.", "body_md": "arXiv:2607.24434v1 Announce Type: cross\nAbstract: Large Mixture-of-Experts (MoE) language models are attractive for end-device deployment because only a small subset of experts is active per token, but their routed expert weights often exceed accelerator memory. We target latency-critical single-user settings where routed experts are staged on demand from CPU memory to a GPU or from Flash to a mobile NPU. In this setting, self-speculative decoding faces a new bottleneck: increasing the draft expert set improves accuracy but triggers extra expert loading, while cheap small-footprint drafts have low acceptance; moreover, verifying a multi-token block activates the union of target experts and is no longer close to one target step. We propose DraftExpert, an expansion-aware self-speculative decoding framework for expert-offloaded MoE inference. DraftExpert trains one lightweight accelerator-resident draft expert per layer by self-distilling residual, logit/token, and router-agreement signals from the frozen target MoE. At inference time, it uses a fixed-footprint shared+top-1+draft-expert drafter together with confidence--expansion truncation and target-expert prefetching, while final tokens are still exactly verified by the target model. On DeepSeek-V2-Lite and Moonlight-16B-A3B across CPU-GPU and Flash-NPU offload, DraftExpert improves decode throughput by 1.45x on average, raises draft acceptance to 84~87%, and achieves 86~88% prefetch hit rates.", "url": "https://wpnews.pro/news/draftexpert-expansion-aware-self-speculative-decoding-for-end-device-moe", "canonical_source": "https://www.machinebrief.com/news/draftexpert-expansion-aware-self-speculative-decoding-for-en-t834", "published_at": "2026-07-28 04:00:00+00:00", "updated_at": "2026-07-28 04:56:59.449794+00:00", "lang": "en", "topics": ["artificial-intelligence", "machine-learning", "large-language-models", "ai-infrastructure"], "entities": ["DraftExpert", "DeepSeek-V2-Lite", "Moonlight-16B-A3B"], "alternates": {"html": "https://wpnews.pro/news/draftexpert-expansion-aware-self-speculative-decoding-for-end-device-moe", "markdown": "https://wpnews.pro/news/draftexpert-expansion-aware-self-speculative-decoding-for-end-device-moe.md", "text": "https://wpnews.pro/news/draftexpert-expansion-aware-self-speculative-decoding-for-end-device-moe.txt", "jsonld": "https://wpnews.pro/news/draftexpert-expansion-aware-self-speculative-decoding-for-end-device-moe.jsonld"}}