Context Language Models (CLMs)

Researchers from the University of Washington, Meta Superintelligence Labs, MIT and Trillium Labs introduced Context Language Models (CLMs), which treat context as a file the model can update itself, reporting 11.4% higher accuracy with 21.5% fewer FLOPs on BrowseComp-Plus and 5% higher scores with 59% fewer FLOPs on 12-hour EdgeBench versus state-of-the-art context-management strategies. The team also reported an online reinforcement learning method that improved Qwen3.5-9B on BrowseComp-Plus by 47.6% with 12% fewer FLOPs, and released code including the clm-harbor CLI, a Pi agent plugin, and a SGLang KV-cache reuse patch under CC BY-NC 4.0.

Rulin Shao https://rulinshao.github.io/ <sup 1,2</sup , Shannon Zejiang Shen https://www.szj.io/ <sup 3</sup , Junjie Oscar Yin https://oseyincs.io/ <sup 1,2</sup , Yuetai Li https://yuetl9.github.io/ <sup 1</sup , Minheng Wang https://minhengwang.github.io/ <sup 1</sup , Hamish Ivison https://ivison.id.au <sup 1</sup , Radha Poovendran https://people.ece.uw.edu/radha/ <sup 1</sup , Nathan Lambert https://natolambert.com/ <sup 4</sup , Teng Xiao https://tengxiao1.github.io/ <sup 1</sup , Mike Lewis https://ai.meta.com/people/209431298931133/mike-lewis/ <sup 2</sup , Wen-tau Yih https://scottyih.org/ <sup 2</sup , Luke Zettlemoyer https://homes.cs.washington.edu/~lsz/ <sup 1,2</sup , Pang Wei Koh https://koh.pw/ <sup 1</sup <sup 1</sup University of Washington <sup 2</sup Meta Superintelligence Labs <sup 3</sup MIT <sup 4</sup Trillium Labs We introduce Context Language Models CLMs , language models that natively manage their own context. We implement this by treating the context as a file and allowing the model to make unrestricted updates to this file. This allows the model to learn what is most important to maintain in context, and naturally extends to multi-agent systems where multiple agent contexts coexist as files. - Zero-shot. Building CLMs zero-shot with existing models outperforms SOTA context-management strategies across a variety of tasks: 11.4% higher accuracy with 21.5% fewer FLOPs on BrowseComp-Plus, 5% higher scores with 59% fewer FLOPs on 12-hour EdgeBench, and 65% greater improvement with the same compute on a 24-hour multi-repository agent-swarm task. - In-context learning. We show that CLMs can be steered with natural-language instructions evolved through a standard skill-optimization loop, improving held-out accuracy by up to 35.9 points on a context-management task while reducing compute. - Reinforcement learning. We also introduce an online reinforcement learning method for CLMs, improving Qwen3.5-9B performance on BrowseComp-Plus by 47.6% while using 12% fewer FLOPs. Run the minimal CLM agent on any Harbor https://github.com/laude-institute/harbor task: pip install -e . clm-harbor run -p <harbor-task -a clm-minimal -m openai/<model \ --agent-kwarg api base=http://localhost:8000/v1 clm-harbor is the Harbor CLI with CLM available as -a clm-minimal . See clm/clm harness https://github.com/facebookresearch/context-language-models/blob/main/clm/clm harness for configuration and serving. Day 1 support: pi-clm for Pi agent https://github.com/earendil-works/pi pi install npm:@lolipopshock/pi-clm | clm/clm harness https://github.com/facebookresearch/context-language-models/blob/main/clm/clm harness | CLM implemented in Harbor https://github.com/laude-institute/harbor | | clm/clm icl https://github.com/facebookresearch/context-language-models/blob/main/clm/clm icl | skill evolution | | clm/clm rl https://github.com/facebookresearch/context-language-models/blob/main/clm/clm rl | reinforcement learning | | suffix cache reuse https://github.com/facebookresearch/context-language-models/blob/main/suffix cache reuse | Suffix Cache Reuse: KV-cache reuse for CLM serving, as a patch to SGLang | - ContextBench If you find our work helpful, we would appreciate it if you could cite our paper: @article{shao2026context, title = {Context Language Models}, author = {Shao, Rulin and Shen, Shannon Zejiang and Yin, Junjie Oscar and Li, Yuetai and Wang, Minheng and Ivison, Hamish and Poovendran, Radha and Lambert, Nathan and Xiao, Teng and Lewis, Mike and Yih, Wen-tau and Zettlemoyer, Luke and Koh, Pang Wei}, journal = {arXiv preprint arXiv:2609.37725}, year = {2026} } This project is licensed under CC BY-NC 4.0 https://github.com/facebookresearch/context-language-models/blob/main/LICENSE . See also NOTICE https://github.com/facebookresearch/context-language-models/blob/main/NOTICE .