{"slug": "context-language-models", "title": "Context Language Models", "summary": "A paper submitted to arXiv on 29 Sep 2026 introduces Context Language Models (CLMs), language models that natively manage their own context by treating the context as a file the model can update without restriction. Built zero-shot with existing models, CLMs beat state-of-the-art context management strategies with 11.4% higher accuracy and 21.5% fewer FLOPs on BrowseComp-Plus, 5% higher scores and 59% fewer FLOPs on 12-hour EdgeBench, and 65% greater improvement at the same compute on a 24-hour multi-repository agent-swarm task. The authors also report an online reinforcement learning method that improves Qwen3.5-9B on BrowseComp-Plus by 47.6% with 12% fewer FLOPs, and a Suffix Cache Reuse co-design for CLM serving that cuts server-side compute 35% versus standard SGLang at matched performance.", "body_md": "# Computer Science > Artificial Intelligence\n\n  [Submitted on 29 Sep 2026]\n\n# Title:Context Language Models\n\n[View PDF](https://arxiv.org/pdf/2609.37725)\n\n[HTML (experimental)](https://arxiv.org/html/2609.37725v1)\n\nAbstract:We introduce Context Language Models (CLMs), language models that natively manage their own context. We implement this by treating the context as a file and allowing the model to make unrestricted updates to this file. This allows the model to learn what is most important to maintain in context, and naturally extends to multi-agent systems where multiple agent contexts coexist as files. Building CLMs zero-shot with existing models outperforms SOTA context management strategies across a variety of tasks: 11.4% higher accuracy with 21.5% fewer FLOPs on BrowseComp-Plus, 5% higher scores with 59% fewer FLOPs on 12-hour EdgeBench, and 65% greater improvement with the same compute on a 24-hour multi-repository agent-swarm task. Moreover, by shifting context management from external harness control to intrinsic model behavior, CLMs naturally enable both in-context and parametric learning of context-management strategies. We show that CLMs can be steered with natural-language instructions evolved through a standard skill-optimization loop, improving held-out accuracy by up to 35.9 points on a context-management task while reducing compute. We also introduce an online reinforcement learning method for CLMs, improving Qwen3.5-9B performance on BrowseComp-Plus by 47.6% while using 12% fewer FLOPs. Finally, we co-design Suffix Cache Reuse for CLM serving, further reducing server-side compute by 35% relative to standard SGLang at matched performance.\n    \n\n### Current browse context:\n\ncs.AI\n\n### References & Citations\n\nLoading...\n\n# Bibliographic and Citation Tools\n\nBibliographic Explorer \n\n*(*[What is the Explorer?](https://info.arxiv.org/labs/showcase.html#arxiv-bibliographic-explorer))\nConnected Papers \n\n*(*[What is Connected Papers?](https://www.connectedpapers.com/about))\nLitmaps \n\n*(*[What is Litmaps?](https://www.litmaps.co/))\nscite Smart Citations \n\n*(*[What are Smart Citations?](https://www.scite.ai/))\n# Code, Data and Media Associated with this Article\n\nalphaXiv \n\n*(*[What is alphaXiv?](https://alphaxiv.org/))\nCatalyzeX Code Finder for Papers \n\n*(*[What is CatalyzeX?](https://www.catalyzex.com))\nDagsHub \n\n*(*[What is DagsHub?](https://dagshub.com/))\nGotit.pub \n\n*(*[What is GotitPub?](http://gotit.pub/faq))\nHugging Face \n\n*(*[What is Huggingface?](https://huggingface.co/huggingface))\nScienceCast \n\n*(*[What is ScienceCast?](https://sciencecast.org/welcome))\n# Demos\n\n# Recommenders and Search Tools\n\nInfluence Flower \n\n*(*[What are Influence Flowers?](https://influencemap.cmlab.dev/))\nCORE Recommender \n\n*(*[What is CORE?](https://core.ac.uk/services/recommender))\n# arXivLabs: experimental projects with community collaborators\n\narXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.\n\nBoth individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.\n\nHave an idea for a project that will add value for arXiv's community? [**Learn more about arXivLabs**](https://info.arxiv.org/labs/index.html).", "url": "https://wpnews.pro/news/context-language-models", "canonical_source": "https://arxiv.org/abs/2609.37725", "published_at": "2026-10-01 14:51:33+00:00", "updated_at": "2026-10-01 19:14:36.989672+00:00", "lang": "en", "topics": ["artificial-intelligence", "large-language-models", "ai-research", "ai-agents", "ai-infrastructure"], "entities": ["arXiv", "Context Language Models", "BrowseComp-Plus", "EdgeBench", "Qwen3.5-9B", "SGLang"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/context-language-models", "markdown": "https://wpnews.pro/news/context-language-models.md", "text": "https://wpnews.pro/news/context-language-models.txt", "jsonld": "https://wpnews.pro/news/context-language-models.jsonld"}}