{"slug": "risa-recoverable-sparse-attention-for-block-diffusion-language-models", "title": "RISA: Recoverable Sparse Attention for Block-Diffusion Language Models", "summary": "Researchers introduced RISA, a recoverable sparse-attention method for block-wise diffusion language models that reuses a shared, bounded read plan across denoising steps instead of rebuilding sparse support each step. On DreamReasoner-8B, RISA retained comparable task performance while speeding up attention 2.67x at 32K context and 1.36x for fixed-trajectory requests versus dense attention, with attention speedup reaching 5.53x at 128K. Across eight settings on three models, RISA decoded faster than the tested MAGE, QUEST and LoSA budgets at matched KL.", "body_md": "arXiv:2611.22191v1 Announce Type: new \nAbstract: Block-wise diffusion language models (DLMs) refine multiple tokens in parallel, offering an alternative to token-by-token autoregressive decoding. Their long-context inference, however, remains bottlenecked by memory-bound attention: every denoising step repeatedly reads a growing historical key--value (KV) cache. Directly applying sparse attention does not fully solve this problem, because different queries can select different pages, inflating the block's collective read set, while repeated support construction adds overhead and permanent eviction can discard information needed later. We present RISA, a recoverable sparse-attention method with shared read plans for block-wise DLMs. RISA exploits the fact that historical KV remains fixed within a block while queries evolve across denoising steps. Newly generated history is organized into exact priority and recovery tiers; seed queries then build a protected, bounded read plan whose indices are reused across the block while attention is recomputed with fresh queries and current-block KV. This separates history preservation from repeated reading and amortizes sparse-support construction without changing the backbone or decoder. On DreamReasoner-8B, RISA retains comparable task performance. At 32K, the default configuration speeds up attention by $2.67\\times$ and fixed-trajectory requests by $1.36\\times$ over dense, and attention speedup reaches $5.53\\times$ at 128K. Across eight settings on three models, RISA decodes faster than the tested MAGE, QUEST, and LoSA budgets at matched KL.", "url": "https://wpnews.pro/news/risa-recoverable-sparse-attention-for-block-diffusion-language-models", "canonical_source": "https://www.machinebrief.com/news/risa-recoverable-sparse-attention-for-block-diffusion-langua-m09x", "published_at": "2026-11-18 05:00:00+00:00", "updated_at": "2026-11-18 05:16:20.287819+00:00", "lang": "en", "topics": ["artificial-intelligence", "machine-learning", "large-language-models", "ai-research", "ai-infrastructure"], "entities": ["RISA", "DreamReasoner-8B", "MAGE", "QUEST", "LoSA", "arXiv"], "also_reported_by": [], "takeaway": "RISA reuses one bounded, protected read plan built from seed queries across all denoising steps of a block, cutting attention time 2.67x at 32K and 5.53x at 128K on DreamReasoner-8B versus dense attention at comparable task performance.", "recommendation": {"advice": "For block-diffusion long-context inference, build a shared sparse read plan from seed queries and reuse it across denoising steps; the source measured 2.67x attention speedup at 32K and 5.53x at 128K.", "playbook": "inference-serving", "playbook_url": "https://wpnews.pro/playbooks/inference-serving"}, "alternates": {"html": "https://wpnews.pro/news/risa-recoverable-sparse-attention-for-block-diffusion-language-models", "markdown": "https://wpnews.pro/news/risa-recoverable-sparse-attention-for-block-diffusion-language-models.md", "text": "https://wpnews.pro/news/risa-recoverable-sparse-attention-for-block-diffusion-language-models.txt", "jsonld": "https://wpnews.pro/news/risa-recoverable-sparse-attention-for-block-diffusion-language-models.jsonld"}}