{"slug": "mask-guided-kv-cache-eviction-in-block-diffusion-language-models", "title": "Mask-Guided KV Cache Eviction in Block Diffusion Language Models", "summary": "A training-free method called MaskAhead reduces key-value cache memory in block diffusion language models by 9.5x on average on long-prompt question answering, at a cost of 1.2 points of mean F1 versus dense inference, according to the arXiv paper 2610.06996v1. The method's quantized variant, Q-MaskAhead, raises the memory reduction to 20.1x with a 2.3-point mean F1 loss, and in a batch-32 systems profile MaskAhead delivers 1.23x end-to-end and 1.68x decode-stage speedups over dense inference. MaskAhead ranks KV entries by estimated contribution to the attention output, using current-block masks to guide selection and probes of upcoming masked blocks to guide eviction, tested on Fast-dLLM-v2, DreamReasoner, and LLaDA2.0-mini.", "body_md": "arXiv:2610.06996v1 Announce Type: new \nAbstract: Block diffusion language models keep a large key-value (KV) cache throughout generation and attend to it at every denoising step, limiting both memory capacity and generation speed. Reducing these costs requires deciding which past tokens to use for denoising the current block (selection) and which to keep in memory for future blocks (eviction). We propose MaskAhead, a training-free method that solves both tasks with a single mask-query-based ranking mechanism. Current-block masks guide selection, while probes of upcoming masked blocks guide eviction. Both rank KV entries by their estimated contribution to the attention output. Our quantized variant, Q-MaskAhead, computes selection and attention directly from low-bit KV, largely preserving the selected entries. Experiments on Fast-dLLM-v2, DreamReasoner, and LLaDA2.0-mini cover long-generation reasoning, long-prompt question answering, and needle-in-a-haystack retrieval. On long-prompt QA, MaskAhead reduces KV memory by $9.5\\times$ on average with a 1.2-point mean F1 loss relative to dense inference. Q-MaskAhead increases the reduction to $20.1\\times$ with a 2.3-point mean F1 loss. In a batch-32 systems profile, MaskAhead achieves $1.23\\times$ end-to-end and $1.68\\times$ decode-stage speedups over dense inference.", "url": "https://wpnews.pro/news/mask-guided-kv-cache-eviction-in-block-diffusion-language-models", "canonical_source": "https://arxiv.org/abs/2610.06996", "published_at": "2026-10-07 04:00:00+00:00", "updated_at": "2026-10-07 04:17:16.100271+00:00", "lang": "en", "topics": ["large-language-models", "ai-research", "ai-infrastructure", "machine-learning"], "entities": ["MaskAhead", "Q-MaskAhead", "Fast-dLLM-v2", "DreamReasoner", "LLaDA2.0-mini", "arXiv"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/mask-guided-kv-cache-eviction-in-block-diffusion-language-models", "markdown": "https://wpnews.pro/news/mask-guided-kv-cache-eviction-in-block-diffusion-language-models.md", "text": "https://wpnews.pro/news/mask-guided-kv-cache-eviction-in-block-diffusion-language-models.txt", "jsonld": "https://wpnews.pro/news/mask-guided-kv-cache-eviction-in-block-diffusion-language-models.jsonld"}}