On this page #
When agents forget after compaction, blame lost state over token ratios. How eight production harnesses manage context and keep long runs alive.
Most developers treat context compaction as a summarization algorithm.
When the agent nears its token limit, the harness calls an LLM, asks for a summary of past turns, and pastes that summary into the next prompt. The team sees a 70% drop in tokens and moves on.
Then the agent enters an infinite loop. It modifies a file it was forbidden to touch. It tests a broken hypothesis it already discarded thirty minutes earlier. It hallucinates an API route because the original schema was wiped from the transcript.
That mental model is wrong. Treat compaction as a pipeline that manages context and rebuilds state. The summary prompt is one step in it.
When an agent fails after compaction, check four things before you blame the compression ratio. Did the harness wipe negative constraints? Lose the record of rejected hypotheses? Skip re project rules? Invalidate the prompt cache?
I looked at eight harnesses: Claude Code, Codex, Antigravity CLI (formerly Gemini CLI), OpenCode, Pi, OMP, OpenHands, and LangChain Deep Agents.
Three Tiers of State #
A reliable agent system keeps three layers of state apart:
- The Durable Record: The complete event log: every user message, tool call, file snapshot, and error trace. The harness keeps it on disk in SQLite, a JSONL session tree, or an append-only event stream.
- The Working Context: The tokens the harness sends to the LLM on turn N. The model window and your budget cap its size.
- The Continuation State: The structured task data the agent needs to resume: objectives, negative constraints, accepted decisions, rejected attempts, and test results.
Compaction rewrites the working context and leaves the durable record on disk.
When a developer says “the agent forgot the conversation,” the working context lost it. Storage still has it. OpenHands builds a new model-facing view and keeps the event stream. OMP keeps the full conversation tree on disk. LangChain Deep Agents writes summarized history to the thread filesystem, where the agent can search it.
Nine Context Reduction Techniques
Harnesses mix nine techniques:
| Technique | Operation | Primary Trade-off |
|---|---|---|
| Truncation | Drop older turns without saving meaning | Fast and cheap, but destroys task history |
| Observation Clearing | Strip raw tool outputs; keep tool calls | Retains intent; loses specific test data |
| Deduplication | Remove superseded file reads or searches | Saves space; requires tracking staleness |
| Externalization | Replace raw text with a disk/URI pointer | Preserves access; requires explicit retrieval |
| Semantic Summarization | Condense dialogue into an LLM narrative | Compact; lossy and prone to hallucination |
| Tail Retention | Keep the last N tokens or turns verbatim | Protects immediate flow; limits space freed |
| Native Provider Compaction | Delegate state reduction to the provider | Low client overhead; vendor lock-in |
| Context Reconstruction | Reload instructions, plans, and files post-summary | Restores truth; refills the context window |
| Context Isolation | Run heavy exploration in subagents | Keeps parent clean; adds IPC orchestration |
How Each Harness Handles a Full Window #
Each harness trades token savings against execution safety and implementation effort:
| Harness | Core Mechanism | What Survives Verbatim | Recovery Mechanism |
|---|---|---|---|
| Claude Code | Output clearing + summary + reconstruction | Root rules, memory, plan, up to 5 files | Direct disk re-reads |
| Codex | 20k token user-message budget + local summary | User messages (up to 20k tokens) | None (older tool outputs lost) |
| Antigravity CLI | Checkpoint compression fires at 70% of the window | Latest 30% of history | /rewind checkpoints and/resume sessions |
| OpenCode | Local summaries (default) or native provider | Token-budgeted recent tail (15k in V2) | Provider checkpoint or disk log |
| Pi | Reserve headroom trigger + protected recent tail | Last 20,000 tokens unsummarized | Extensions or branch rewinds |
| OMP | Fallback chain ( remote →snapcompact →handoff →shake →soft ) |
Method-specific recent tail | artifact:// URIs and image recall |
| OpenHands | Append-only event log + Condenser pipeline | Head prefix and recent tail | Durable event stream replay |
| Deep Agents | Tool off to filesystem + 85% limit trigger | Last 10% of window tokens | Thread filesystem file reads |
Claude Code: The Context Reconstruction Pipeline #
Claude Code rebuilds more state after compaction than any other harness I looked at.
It runs five stages:
What Survives in Claude Code
Claude Code’s context window documentation lists what survives:
- Root
CLAUDE.mdand global rules: Re-read from disk on every compaction pass. - Auto memory: Re-injected from
~/.claude/projects/<project>/memory/. - Plan-mode tasks: The active plan markdown file is re-injected.
- Active working set: Claude Code re-reads up to five files it recently read or edited, newest first. For a file over 5,000 tokens, it inserts a path reference instead of the contents.
- Skill definitions: Claude Code caps inlined skill content at 5,000 tokens per skill and 25,000 tokens across all skills.
- Path-scoped rules: Claude Code drops them and re-reads them when the agent touches a matching file.
The practical rule: Put any instruction that must survive compaction in CLAUDE.md or a skill file. A rule you state in chat survives only if the summarizer keeps it.
The Compaction Thrashing Hazard
Claude Code’s docs name a failure mode called compaction thrashing.
Compaction can only free tokens you have not pinned. Take an agent in a large monorepo with a 160,000-token compaction threshold:
- The fixed load is heavy: system prompt and tool schemas from a dozen MCP servers (60,000 tokens), plus
CLAUDE.mdfiles and rules (30,000 tokens). - Compaction triggers. The summarizer compresses the conversation into 10,000 tokens.
- The reconstruction step re-reads five working files from disk (+25,000 tokens).
- The new floor is 125,000 tokens. Only 35,000 tokens of headroom remain.
- The agent runs a test suite that prints 40,000 tokens of error traces. The window crosses the threshold again on the next turn.
Without strict token limits on reconstruction, the agent spends more time compacting and rebuilding state than writing code. Claude Code watches for this. If context refills right after each summary, it stops auto-compacting after a few attempts and shows an error.
Tail Budgets and Protected Channels: Codex, Pi, and OpenCode #
Some harnesses give specific message types their own token budget instead of treating history as one pool.
Codex: The 20,000-Token User Budget
The Codex source reserves a 20,000-token budget for user messages.
When compaction runs:
- The harness selects user messages by walking backward from the most recent turn.
- It accumulates user messages until reaching 20,000 tokens.
- It reverses those messages back into chronological order.
- It appends the generated summary of the assistant and tool interactions.
Your prompts, constraints, and feedback stay word for word. Codex compresses the tool output around them.
If the summarization call itself overflows the context limit, Codex drops the oldest history item and retries at once. It keeps trimming until the call fits and errors out when one item is left. Codex uses exponential backoff for other transient errors.
Pi: Two Numbers
Pi exposes two settings:
reserve: Tokens Pi keeps free for the next response (default: 16,384).recent: Tokens Pi keeps verbatim (default: 20,000).
The trigger:
Pi compacts when total context tokens exceed the window limit minus the reserve :
Pi’s extension architecture separates context maintenance into three questions:
| Interface | Question It Answers | Example Answer |
|---|---|---|
| Trigger policy | When should maintenance run? | Token threshold crossed |
| Compaction engine | What replaces older history? | Local or remote summary |
| Commit policy | When is the swap safe? | Only at a turn boundary |
Splitting the trigger from the commit policy stops Pi from cutting a tool turn in half.
OpenCode: V1 Pruning to V2 Policies
OpenCode shows how a context strategy changes between versions.
In V1, OpenCode pruned tool output alongside summarization:
- Protected the most recent 40,000 tokens.
- Required a minimum removal threshold of 20,000 tokens before committing a prune.
- Excluded certain tools (such as
skill) from being pruned. - Clamped individual tool result strings to 2,000 characters before sending them to the summarizer.
In OpenCode V2, those hard-coded constants became configuration:
| Concern | V1 (hard-coded) | V2 (config) |
|---|---|---|
| Retained token budget | PRUNE_PROTECT = 40_000 |
compaction.keep.tokens (default 15,000) |
| Compaction headroom | fixed | compaction.buffer (15% of the limit, at least 16k) |
| Provider engine | Local summaries only | compaction.type :"summary" or"native" , per provider or model |
| hiding | none | experimental.background_compaction starts the summary at 85% of the threshold |
The 2,000-character clamp stops one giant JSON dump from overflowing the summarizer prompt. It also hides data. If the key failure line in a stack trace sits at character 2,500, the summarizer never sees it.
Three Other Approaches: Images, Files, and Event Views #
A second LLM call to summarize text costs time and money, and it loses detail. OMP, Deep Agents, and OpenHands each try something else.
OMP: Snapcompact and the Maintenance Fallback Chain
OMP implements an ordered chain of maintenance strategies:
OMP’s oddest strategy is Snapcompact.
Snapcompact changes the format and keeps the content. It renders older turns into dense images with a bundled image font and attaches them to the prompt as vision inputs.
Why it works: Vision models bill an image by its pixel size, whatever the amount of text inside. A page of dense terminal logs can cost far fewer tokens as an image than as text.
The trade-off: Models misread exact characters in dense images. Indentation, small punctuation, and 40-character commit hashes come back wrong. Snapcompact skips the summary call, but treat what it keeps as lossy.
For text-based reduction, OMP provides /shake elide. It replaces command output, search results, file reads, and large fenced blocks with placeholders. In persisted sessions, those placeholders carry an artifact:// URI where possible. If the model needs the data later, it reads the artifact.
LangChain Deep Agents: Off to the Filesystem
LangChain Deep Agents follows one rule: offload before you summarize.
When a tool result exceeds 20,000 tokens, the harness writes the full output to the filesystem under /large_tool_results/. The model sees a file path and a short preview:
Deep Agents treats large file writes the same way, later. Once the session passes 85% of the window, it cuts older write and edit calls down to a pointer to the file on disk. The code already sits in the workspace, so 2,000 lines of it in the history buy you nothing.
Summarization fires at 85% of the model’s window by default. Deep Agents keeps the most recent 10% and writes the summarized turns to /conversation_history/, where the agent can search them with its normal file tools.
OpenHands: Event Log and Condensers
OpenHands keeps storage and the model’s view apart.
It records every action and observation in an append-only event log. The model reads a View that a pipeline of Condenser objects builds from that log:
When context fills, the condenser emits a Condensation event that records the range of forgotten event IDs and injects a summary at that offset.
That makes debugging easy. You can replay a session, audit what the condenser removed, or run two compaction algorithms against the same event stream and compare them.
The risk: The LLMSummarizingCondenser was built around event counts (max_size, 240 events by default in the SDK source). Event count is a poor proxy for token pressure. One tool call returning a 100KB JSON payload can outweigh 200 short turns. The SDK now adds a max_tokens trigger, and the model’s input limit acts as a hard ceiling. Trigger on rendered tokens.
Antigravity CLI: Protocol Safety
Antigravity CLI (Google’s successor to Gemini CLI, executable as agy) compresses chat history once it reaches 70% of the model’s limit (model.chatCompression.contextPercentageThreshold) and keeps the latest 30% verbatim. Function responses get a separate 50,000-token budget during compression. Antigravity CLI also refuses to break tool protocol state.
Model APIs require a matching tool response right after each function call. If a compressor strips an old tool response and keeps the call, the provider returns a 400 error.
Antigravity CLI’s client code blocks context updates between an outstanding tool call and its response. Whatever compresses your context has to follow the API’s message rules.
The Hidden Cost: Compaction Breaks Your Prompt Cache #
Teams assume compaction saves money because it cuts tokens. With prompt caching, compaction can raise your bill.
Anthropic, OpenAI, and Google rely on prefix caching. If the first 80,000 tokens of your prompt match the previous turn byte for byte, the provider bills them at a discount: about 90% off on most current Anthropic, OpenAI, and Gemini models, and up to 95–97.5% off on the newest ones.
Compaction rewrites the middle of the prompt, so the cache misses from the first changed byte onward:
A compaction event costs more than the summary tokens:
| Term | What You Pay For |
|---|---|
| The summary call: it reads the whole history it replaces | |
| The first turn after compaction, billed at cache-write rates | |
| Re-reading files, plans, and rules into the new prompt | |
| Work the agent redoes because the summary lost something |
Run the numbers with Anthropic’s standard rates (cache reads at 0.1x, 5-minute cache writes at 1.25x). A warm 100,000-token prefix costs the equivalent of 10,000 input tokens per turn. Compact it to 70,000 tokens, and the first turn after compaction pays about 1.25 × 60,000 = 75,000 for everything after the system prompt. That one turn costs about 7x a normal cached turn, before you count the summary call. You see savings on later turns, once they reuse the smaller prefix. If your harness compacts too often, it never earns the cost back.
Latency: Measure the Whole
Measure the whole the user sees, from the start of pruning to the next turn’s first token:
OMP (compaction.asyncEnabled, on by default) and OpenCode V2 ( experimental.background_compaction) both do this. They prepare the summary shortly before the threshold, so when the boundary hits, the summary is ready to install.
Provider Compaction vs Harness Compaction #
Should you use OpenAI’s native /responses/compact or build compaction in your harness?
OpenAI’s Responses API offers two modes: automatic server-side compaction and an explicit /responses/compact endpoint. Both return an opaque, encrypted compaction item.
| Feature | Local Harness Compaction | Native Provider Compaction |
|---|---|---|
| Inspectability | Fully readable text and JSON | Encrypted opaque blob |
| Editability | Developers can inject or edit state | Immutable black box |
| Model Portability | Works across Anthropic, OpenAI, Gemini | Locked to specific provider models |
| Storage | Stored in local SQLite or JSONL | Managed on provider servers |
| Cache Behavior | Harness controls cache alignment | Optimized provider-side |
The rule: Compact in your harness when you need to inspect state, run more than one model, or build your own recovery tools. Use provider compaction for single-vendor apps where shipping speed matters more than visibility.
Six Ways Compaction Breaks an Agent #
Test your harness for these six failure modes:
| Failure Mode | How It Manifests | Engineering Mitigation |
|---|---|---|
| Summary Drift | Repeated compactions gradually alter the task objective | Anchor the root objective in an immutable prompt slot |
| Lost Negative Constraints | ”Do not modify auth.ts” disappears from the summary | Store negative constraints in structured metadata |
| Repeating Rejected Hypotheses | Agent tests an approach that failed an hour ago | Preserve an explicit rejected_approaches list |
| Stale Reconstruction | Reloaded file contents desync from the summarized state | Record file content hashes and timestamps at compaction |
| Broken Tool Sequences | Orphaned tool calls cause API 400 validation errors | Compact only at completed tool-turn boundaries |
| Compaction Thrashing | Large reconstructed context re-triggers compaction on the next turn | Enforce strict token budgets on reconstructed files |
The Production Context Pipeline: Structured State Over Prose #
If you build an agent harness, stop asking models for free-form narrative summaries. Prose is ambiguous and drops detail.
Have the model write structured continuation state instead.
The Continuation Checkpoint Schema
At compaction, have the model emit JSON in a strict schema:
{
"task_objective": "Fix webhook duplicate delivery in payment service",
"negative_constraints": [
"Do not modify the public v1 API signature",
"Do not add external Redis dependencies"
],
"accepted_decisions": [
"Use SQLite unique constraint on idempotency_key",
"Return HTTP 200 on duplicate event without re-processing"
],
"rejected_hypotheses": [
{
"approach": "In-memory LRU cache",
"reason": "Process restarts drop state and allow duplicate charges"
}
],
"completed_milestones": [
"Added migration for idempotency_records table",
"Implemented checkIdempotency helper"
],
"pending_work": [
"Update stripeWebhookHandler to catch UniqueConstraintError",
"Run integration test suite"
],
"verification": {
"passed": ["tests/unit/idempotency.test.ts"],
"failed": [],
"not_run": ["tests/e2e/webhook.test.ts"]
},
"external_artifacts": [
{
"name": "Stripe webhook failure payload",
"uri": "artifact://webhook_payload_982.json"
}
],
"next_immediate_action": "Edit src/handlers/stripe.ts around line 45"
}
The fields that matter most are negative_constraints, rejected_hypotheses, and verification.
Prose summaries mangle these fields first. Restore the JSON checkpoint into the prompt, and the agent knows what to avoid, what failed, and which checks it still has to run.
Separate Your Context Budgets
Give each layer its own token budget:
With separate budgets, you can see what filled the window: tool schemas, file rehydration, or chat history.
How to Benchmark Your Compaction Engine #
Compression ratio alone tells you little. An engine that saves 95% of tokens by deleting every negative constraint is useless.
Build a test suite around ten failure scenarios:
- The Buried Negative Constraint: Give the agent a rule on Turn 1 (“Never edit files in
/config”). Force three compaction cycles across 50 tool calls. Check if the agent modifies/config. - The Failed Hypothesis Trap: Let the agent attempt a broken fix that fails tests. Force compaction. Check if the agent attempts the exact same fix again.
- The Stale File Edit: Have an external process modify a file on disk after the agent read it. Force compaction. Check if the rehydration step catches the new file hash.
- The Unique Observation Test: Provide an error log containing a unique UUID or port number. Clear the tool output. Verify the agent can retrieve the exact UUID using its artifact tool.
- The Long Tool Turn: Generate a single tool response with 80,000 tokens of JSON. Verify the harness offloads or clamps the output without crashing the summarizer.
- The Running Background Worker: Start a long-running test server. Force compaction. Verify the agent does not attempt to launch a duplicate server on the same port.
- Cache Cost Accounting: Calculate the total billed cost across 20 turns with compaction enabled versus baseline runs. Measure whether cache misses wiped out your token savings.
- The Model Switch Test: Perform compaction on an Anthropic model, then switch execution to an OpenAI model. Verify the continuation state loads without provider-specific syntax errors.
- The Protocol Sequence Invariant: Interrupt the agent mid-turn during an outstanding tool call. Force compaction. Verify the resulting prompt has valid function call and response pairing.
- The Repeated-Cycle Stability Test: Run an agent through six consecutive compaction cycles on a complex refactoring task. Measure whether task completion degrades over time.
The Bottom Line #
Treat context compaction as an operating system service for your agents.
Free-form text summaries will keep breaking long tasks. The harnesses that survive long runs follow three rules:
- Keep the durable record intact on disk. Compaction should never delete your event log.
- Offload heavy outputs before summarizing. Replace large file reads and tool outputs with disk pointers and artifact URIs.
- Restore structured continuation state. Keep negative constraints, rejected hypotheses, and test statuses as typed JSON.
Judge a harness by how faithfully it resumes work after compaction. How much text it squeezes out matters less.
Building AI agent harnesses or context engineering pipelines? I would love to hear how your team handles context exhaustion and state recovery. Reach out on LinkedIn.