# Context Language Models (CLMs)

> Source: <https://github.com/facebookresearch/context-language-models>
> Published: 2026-09-30 22:54:50+00:00

[Rulin Shao](https://rulinshao.github.io/)<sup>1,2</sup>,
  [Shannon Zejiang Shen](https://www.szj.io/)<sup>3</sup>,
  [Junjie Oscar Yin](https://oseyincs.io/)<sup>1,2</sup>,
  [Yuetai Li](https://yuetl9.github.io/)<sup>1</sup>,
  [Minheng Wang](https://minhengwang.github.io/)<sup>1</sup>,
  [Hamish Ivison](https://ivison.id.au)<sup>1</sup>,
  [Radha Poovendran](https://people.ece.uw.edu/radha/)<sup>1</sup>,
  [Nathan Lambert](https://natolambert.com/)<sup>4</sup>,
  [Teng Xiao](https://tengxiao1.github.io/)<sup>1</sup>,
  [Mike Lewis](https://ai.meta.com/people/209431298931133/mike-lewis/)<sup>2</sup>,
  [Wen-tau Yih](https://scottyih.org/)<sup>2</sup>,
  [Luke Zettlemoyer](https://homes.cs.washington.edu/~lsz/)<sup>1,2</sup>,
  [Pang Wei Koh](https://koh.pw/)<sup>1</sup>

<sup>1</sup>University of Washington   <sup>2</sup>Meta Superintelligence Labs   <sup>3</sup>MIT   <sup>4</sup>Trillium Labs

We introduce **Context Language Models (CLMs)**, language models that natively manage their own
context. We implement this by treating the **context as a file** and allowing the model to make
unrestricted updates to this file. This allows the model to learn what is most important to
maintain in context, and naturally extends to multi-agent systems where multiple agent
contexts coexist as files.

- **Zero-shot.** Building CLMs zero-shot with existing models outperforms SOTA
context-management strategies across a variety of tasks: 11.4% higher accuracy with 21.5%
fewer FLOPs on BrowseComp-Plus, 5% higher scores with 59% fewer FLOPs on 12-hour EdgeBench,
and 65% greater improvement with the same compute on a 24-hour multi-repository agent-swarm
task.
- **In-context learning.** We show that CLMs can be steered with natural-language
instructions evolved through a standard skill-optimization loop, improving held-out
accuracy by up to 35.9 points on a context-management task while reducing compute.
- **Reinforcement learning.** We also introduce an online reinforcement learning method for
CLMs, improving Qwen3.5-9B performance on BrowseComp-Plus by 47.6% while using 12% fewer
FLOPs.

Run the minimal CLM agent on any [Harbor](https://github.com/laude-institute/harbor) task:

```
pip install -e .
clm-harbor run -p <harbor-task> -a clm-minimal -m openai/<model> \
  --agent-kwarg api_base=http://localhost:8000/v1
```

`clm-harbor` is the Harbor CLI with CLM available as `-a clm-minimal`. See
[`clm/clm_harness`](https://github.com/facebookresearch/context-language-models/blob/main/clm/clm_harness) for configuration and serving.

## Day 1 support: pi-clm for [Pi agent](https://github.com/earendil-works/pi)

```
pi install npm:@lolipopshock/pi-clm
```

| [`clm/clm_harness`](https://github.com/facebookresearch/context-language-models/blob/main/clm/clm_harness) | CLM implemented in [Harbor](https://github.com/laude-institute/harbor) | 
| [`clm/clm_icl`](https://github.com/facebookresearch/context-language-models/blob/main/clm/clm_icl) | skill evolution | 
| [`clm/clm_rl`](https://github.com/facebookresearch/context-language-models/blob/main/clm/clm_rl) | reinforcement learning | 
| [`suffix_cache_reuse`](https://github.com/facebookresearch/context-language-models/blob/main/suffix_cache_reuse) | Suffix Cache Reuse: KV-cache reuse for CLM serving, as a patch to SGLang | 

- ContextBench

If you find our work helpful, we would appreciate it if you could cite our paper:

```
@article{shao2026context,
  title   = {Context Language Models},
  author  = {Shao, Rulin and Shen, Shannon Zejiang and Yin, Junjie Oscar and Li, Yuetai and
             Wang, Minheng and Ivison, Hamish and Poovendran, Radha and Lambert, Nathan and
             Xiao, Teng and Lewis, Mike and Yih, Wen-tau and Zettlemoyer, Luke and Koh, Pang Wei},
  journal = {arXiv preprint arXiv:2609.37725},
  year    = {2026}
}
```

This project is licensed under [CC BY-NC 4.0](https://github.com/facebookresearch/context-language-models/blob/main/LICENSE). See also [NOTICE](https://github.com/facebookresearch/context-language-models/blob/main/NOTICE).