{"slug": "context-language-models-clms", "title": "Context Language Models (CLMs)", "summary": "Researchers from the University of Washington, Meta Superintelligence Labs, MIT and Trillium Labs introduced Context Language Models (CLMs), which treat context as a file the model can update itself, reporting 11.4% higher accuracy with 21.5% fewer FLOPs on BrowseComp-Plus and 5% higher scores with 59% fewer FLOPs on 12-hour EdgeBench versus state-of-the-art context-management strategies. The team also reported an online reinforcement learning method that improved Qwen3.5-9B on BrowseComp-Plus by 47.6% with 12% fewer FLOPs, and released code including the clm-harbor CLI, a Pi agent plugin, and a SGLang KV-cache reuse patch under CC BY-NC 4.0.", "body_md": "[Rulin Shao](https://rulinshao.github.io/)<sup>1,2</sup>,\n  [Shannon Zejiang Shen](https://www.szj.io/)<sup>3</sup>,\n  [Junjie Oscar Yin](https://oseyincs.io/)<sup>1,2</sup>,\n  [Yuetai Li](https://yuetl9.github.io/)<sup>1</sup>,\n  [Minheng Wang](https://minhengwang.github.io/)<sup>1</sup>,\n  [Hamish Ivison](https://ivison.id.au)<sup>1</sup>,\n  [Radha Poovendran](https://people.ece.uw.edu/radha/)<sup>1</sup>,\n  [Nathan Lambert](https://natolambert.com/)<sup>4</sup>,\n  [Teng Xiao](https://tengxiao1.github.io/)<sup>1</sup>,\n  [Mike Lewis](https://ai.meta.com/people/209431298931133/mike-lewis/)<sup>2</sup>,\n  [Wen-tau Yih](https://scottyih.org/)<sup>2</sup>,\n  [Luke Zettlemoyer](https://homes.cs.washington.edu/~lsz/)<sup>1,2</sup>,\n  [Pang Wei Koh](https://koh.pw/)<sup>1</sup>\n\n<sup>1</sup>University of Washington   <sup>2</sup>Meta Superintelligence Labs   <sup>3</sup>MIT   <sup>4</sup>Trillium Labs\n\nWe introduce **Context Language Models (CLMs)**, language models that natively manage their own\ncontext. We implement this by treating the **context as a file** and allowing the model to make\nunrestricted updates to this file. This allows the model to learn what is most important to\nmaintain in context, and naturally extends to multi-agent systems where multiple agent\ncontexts coexist as files.\n\n- **Zero-shot.** Building CLMs zero-shot with existing models outperforms SOTA\ncontext-management strategies across a variety of tasks: 11.4% higher accuracy with 21.5%\nfewer FLOPs on BrowseComp-Plus, 5% higher scores with 59% fewer FLOPs on 12-hour EdgeBench,\nand 65% greater improvement with the same compute on a 24-hour multi-repository agent-swarm\ntask.\n- **In-context learning.** We show that CLMs can be steered with natural-language\ninstructions evolved through a standard skill-optimization loop, improving held-out\naccuracy by up to 35.9 points on a context-management task while reducing compute.\n- **Reinforcement learning.** We also introduce an online reinforcement learning method for\nCLMs, improving Qwen3.5-9B performance on BrowseComp-Plus by 47.6% while using 12% fewer\nFLOPs.\n\nRun the minimal CLM agent on any [Harbor](https://github.com/laude-institute/harbor) task:\n\n```\npip install -e .\nclm-harbor run -p <harbor-task> -a clm-minimal -m openai/<model> \\\n  --agent-kwarg api_base=http://localhost:8000/v1\n```\n\n`clm-harbor` is the Harbor CLI with CLM available as `-a clm-minimal`. See\n[`clm/clm_harness`](https://github.com/facebookresearch/context-language-models/blob/main/clm/clm_harness) for configuration and serving.\n\n## Day 1 support: pi-clm for [Pi agent](https://github.com/earendil-works/pi)\n\n```\npi install npm:@lolipopshock/pi-clm\n```\n\n| [`clm/clm_harness`](https://github.com/facebookresearch/context-language-models/blob/main/clm/clm_harness) | CLM implemented in [Harbor](https://github.com/laude-institute/harbor) | \n| [`clm/clm_icl`](https://github.com/facebookresearch/context-language-models/blob/main/clm/clm_icl) | skill evolution | \n| [`clm/clm_rl`](https://github.com/facebookresearch/context-language-models/blob/main/clm/clm_rl) | reinforcement learning | \n| [`suffix_cache_reuse`](https://github.com/facebookresearch/context-language-models/blob/main/suffix_cache_reuse) | Suffix Cache Reuse: KV-cache reuse for CLM serving, as a patch to SGLang | \n\n- ContextBench\n\nIf you find our work helpful, we would appreciate it if you could cite our paper:\n\n```\n@article{shao2026context,\n  title   = {Context Language Models},\n  author  = {Shao, Rulin and Shen, Shannon Zejiang and Yin, Junjie Oscar and Li, Yuetai and\n             Wang, Minheng and Ivison, Hamish and Poovendran, Radha and Lambert, Nathan and\n             Xiao, Teng and Lewis, Mike and Yih, Wen-tau and Zettlemoyer, Luke and Koh, Pang Wei},\n  journal = {arXiv preprint arXiv:2609.37725},\n  year    = {2026}\n}\n```\n\nThis project is licensed under [CC BY-NC 4.0](https://github.com/facebookresearch/context-language-models/blob/main/LICENSE). See also [NOTICE](https://github.com/facebookresearch/context-language-models/blob/main/NOTICE).", "url": "https://wpnews.pro/news/context-language-models-clms", "canonical_source": "https://github.com/facebookresearch/context-language-models", "published_at": "2026-09-30 22:54:50+00:00", "updated_at": "2026-09-30 23:19:10.683182+00:00", "lang": "en", "topics": ["artificial-intelligence", "large-language-models", "ai-agents", "ai-research", "ai-tools"], "entities": ["Context Language Models", "University of Washington", "Meta Superintelligence Labs", "MIT", "Trillium Labs", "Rulin Shao", "Qwen3.5-9B", "Harbor"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/context-language-models-clms", "markdown": "https://wpnews.pro/news/context-language-models-clms.md", "text": "https://wpnews.pro/news/context-language-models-clms.txt", "jsonld": "https://wpnews.pro/news/context-language-models-clms.jsonld"}}