{"slug": "specter-fast-moe-inference-via-quantization-enabled-multi-step-speculative", "title": "Specter: Fast MoE Inference via Quantization-Enabled Multi-Step Speculative Offloading", "summary": "Researchers present Specter, an inference system that uses quantization to combine multi-step speculative offloading for mixture-of-experts (MoE) models that exceed a single accelerator's memory capacity. Specter targets the PCIe I/O bottleneck caused by offloading experts to host memory, where data-dependent routing keeps transfers on the critical path.", "body_md": "Mixture-of-experts (MoE) models often exceed a single accelerator's memory capacity. Offloading experts to host memory creates a severe PCIe I/O bottleneck because data-dependent routing keeps transfers on the critical path. We present Specter, an inference system that uses quantization to combine s", "url": "https://wpnews.pro/news/specter-fast-moe-inference-via-quantization-enabled-multi-step-speculative", "canonical_source": "https://aiflash.com/news/136527/", "published_at": "2026-10-21 06:00:42+00:00", "updated_at": "2026-10-21 06:17:59.958284+00:00", "lang": "en", "topics": ["ai-infrastructure", "machine-learning", "large-language-models", "ai-research"], "entities": ["Specter"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/specter-fast-moe-inference-via-quantization-enabled-multi-step-speculative", "markdown": "https://wpnews.pro/news/specter-fast-moe-inference-via-quantization-enabled-multi-step-speculative.md", "text": "https://wpnews.pro/news/specter-fast-moe-inference-via-quantization-enabled-multi-step-speculative.txt", "jsonld": "https://wpnews.pro/news/specter-fast-moe-inference-via-quantization-enabled-multi-step-speculative.jsonld"}}