A new paper suggests agents that compress their history into handoff notes can match full-history performance at a fraction of the context
The AI industry has spent a lot of energy building models that can hold more and more text in memory at once. A new Nvidia paper suggests agents may not need most of it.
The research, titled “You don’t need long context,” found that AI agents can preserve accuracy on complex tasks by regularly compacting their conversation history into short handoff notes. Researchers were able to shrink the working context to as little as 16,000 tokens while still matching full-history performance.
What the researchers actually tested #
The paper is authored by Peter Belcak, Pavlo Molchanov, and nine co-authors. Its focus is frontier agents, meaning AI systems built to work through long, multi-step tasks rather than answer a single question.
Instead of dragging the entire conversation forward at every step, the agent periodically condenses what has happened into a handoff note. The next step starts fresh with only that summary.
The team ran the approach on two models: Anthropic’s Claude Opus 4.5 and OpenAI’s GPT 5.6 Sol. Testing covered thirteen benchmarks. Six of those were long-horizon tasks, including SWE-bench Verified and Terminal-Bench 2.1.
The numbers behind the claim #
The headline result came from Claude Opus 4.5 on SWE-bench Verified, a software engineering benchmark. With one action per context, where the agent hands off after every single step, the model scored 70.2%.
AI, tech, and the markets they move—in one daily briefing.
Daily. Free. Join 34,000+ readers across crypto, finance, and policy.
With its full history available, it scored 70.0%. So the agent that kept forgetting everything except its notes did marginally better than the one that remembered it all.
The researchers also found that tuning the compaction settings could lift Opus 4.5’s score on that task as high as 74.8%.
Across configurations, the accuracy changes were generally modest. For Opus 4.5, shifts ranged from −3.7 to +5.6 points against the full-history baseline. For GPT 5.6 Sol, the range ran from −3.3 to +4.4 points.
In the single-action compaction tests, peak context dropped from 130K–200K tokens to between 15K–37K. Median context sat between 7K–9K tokens.
Why the context window race may be the wrong race #
Nvidia’s researchers argue the better lever is how context gets managed. The paper points to efficient summarization, selective reading of information, and structured handoffs as more effective ways to handle agentic workloads than simply stuffing more tokens in.
What this means for AI builders and their budgets #
The most immediate implication is cost. According to the research, long-context inference can run up to 7x more expensive for models using 256K tokens compared with those using 16K tokens.
Disclosure: This article was edited by Diego Almada Lopez. For more information on how we create and review content, see our Editorial Policy.