{"slug": "language-models-can-control-their-own-attention", "title": "Language Models Can Control Their Own Attention", "summary": "Researchers introduced Declarative Attention (DA), a protocol that lets language models declare which parts of their context they need to attend to, reducing total attended tokens by 52.0% on Gemma-4-31B and 31.1% on Qwen-3.6-27B across 15 long-context tasks, with modest accuracy drops of 1.27 and 2.75 percentage points respectively. The method, described in a paper submitted to arXiv on September 2, 2026, partitions generation into global, focus, and local modes, allowing the inference engine to skip most of the KV cache read.", "body_md": "# Computer Science > Computation and Language\n\n[Submitted on 2 Sep 2026]\n\n# Title:Language Models Can Control Their Own Attention\n\n[View PDF](/pdf/2609.02737)\n\n[HTML (experimental)](https://arxiv.org/html/2609.02737v1)\n\nAbstract:Language models spend most of their attention on a small fraction of context, yet they read the entire KV cache to find the few tokens that matter. If the user asks about a previous detail in a 1M-token conversation, global attention layers must scan the full context to generate each token of the reply. A prominent approach mitigates this cost by pre-selecting relevant tokens via lightweight proxy scores, but this extrinsic scoring still incurs O(N) per step. We take an intrinsic approach motivated by the simple question: wouldn't the model already know which parts of the context are relevant? To this end, we introduce Declarative Attention (DA), a protocol that elicits the model to declare where it needs to attend within its chain-of-thought, partitioning generation into three modes: <global> (full context), <focus> (a specific region), and <local> (recent output only). The inference engine parses these declarations like tool calls and skips most of the KV cache read. Under zero-shot evaluation across 15 long-context tasks, DA on off-the-shelf models (Gemma-4-31B, Qwen-3.6-27B) significantly reduces total attended tokens during decoding (52.0%, 31.1%) with modest accuracy drops (1.27pp, 2.75pp) that shrink with model scale. DA unlocks a new axis of sparse attention, with further potential under training-based methods that future work can explore.\n\n### Current browse context:\n\ncs.CL\n\n### References & Citations\n\nLoading...\n\n# Bibliographic and Citation Tools\n\nBibliographic Explorer\n\n*(*[What is the Explorer?](https://info.arxiv.org/labs/showcase.html#arxiv-bibliographic-explorer))\nConnected Papers\n\n*(*[What is Connected Papers?](https://www.connectedpapers.com/about))\nLitmaps\n\n*(*[What is Litmaps?](https://www.litmaps.co/))\nscite Smart Citations\n\n*(*[What are Smart Citations?](https://www.scite.ai/))# Code, Data and Media Associated with this Article\n\nalphaXiv\n\n*(*[What is alphaXiv?](https://alphaxiv.org/))\nCatalyzeX Code Finder for Papers\n\n*(*[What is CatalyzeX?](https://www.catalyzex.com))\nDagsHub\n\n*(*[What is DagsHub?](https://dagshub.com/))\nGotit.pub\n\n*(*[What is GotitPub?](http://gotit.pub/faq))\nHugging Face\n\n*(*[What is Huggingface?](https://huggingface.co/huggingface))\nScienceCast\n\n*(*[What is ScienceCast?](https://sciencecast.org/welcome))# Demos\n\n# Recommenders and Search Tools\n\nInfluence Flower\n\n*(*[What are Influence Flowers?](https://influencemap.cmlab.dev/))\nCORE Recommender\n\n*(*[What is CORE?](https://core.ac.uk/services/recommender))# arXivLabs: experimental projects with community collaborators\n\narXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.\n\nBoth individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.\n\nHave an idea for a project that will add value for arXiv's community? [ Learn more about arXivLabs](https://info.arxiv.org/labs/index.html).", "url": "https://wpnews.pro/news/language-models-can-control-their-own-attention", "canonical_source": "https://arxiv.org/abs/2609.02737", "published_at": "2026-09-03 02:24:39+00:00", "updated_at": "2026-09-03 02:52:07.727325+00:00", "lang": "en", "topics": ["large-language-models", "ai-research", "ai-infrastructure"], "entities": ["arXiv", "Gemma-4-31B", "Qwen-3.6-27B"], "alternates": {"html": "https://wpnews.pro/news/language-models-can-control-their-own-attention", "markdown": "https://wpnews.pro/news/language-models-can-control-their-own-attention.md", "text": "https://wpnews.pro/news/language-models-can-control-their-own-attention.txt", "jsonld": "https://wpnews.pro/news/language-models-can-control-their-own-attention.jsonld"}}