Context window management is the invisible cost in every multi-step agent pipeline.
I see this pattern across production consulting work. An agent that works cleanly in testing — ten steps, coherent reasoning, correct output — starts degrading six weeks after ship. No errors. No crashes. Just drift. Outputs get shorter. Reasoning steps that used to show clear logic start reading like vague summaries. Users start noticing that the agent "feels slower."
The root cause is almost always the same. Context grows with every step and nobody measured it.
A multi-step agent accumulates its full conversation history by default. Every tool call result gets appended. Every intermediate reasoning step gets stored. The conversation object passed to each model call grows linearly with the number of steps executed.
By step 8 or 9 in a ten-step pipeline, the model is receiving 90% of the context window on each call. Technically within limits, but attention spread thin across thousands of tokens of earlier reasoning that no longer affects the current step.
Here is what this looks like in a real pipeline. Suppose a research agent runs a search, extracts entities, resolves ambiguities, retrieves documents, ranks them, synthesizes a draft, and fact-checks it. That is seven steps. If each step produces an average of 800 tokens of tool output and the model includes its full reasoning trace for each step:
For a 16K context model, that last step is operating with almost nothing left for the actual synthesis task. For a 128K context model, the numbers look more comfortable until you realize you are paying for all of that context on every call, and the model's attention is being diluted across material that stopped being relevant three steps ago. Single-query testing never reveals context saturation. A developer runs the pipeline ten times against a test set, sees correct outputs, ships it. What they tested was ten fresh conversation threads, each starting at zero tokens.
Production load looks nothing like that. Users run sequential tasks. A pipeline that runs five requests across the same session accumulates context from all five runs if the session history is not explicitly bounded. An orchestration layer that retries failed steps passes the failure reasoning back into context. Long-running agents that call external APIs and wait for results accumulate tool logs while they wait.
Context growth under real query volume is not a scaling problem in the usual sense. It is a state management problem. The pipeline has no concept of which historical context is still relevant to the current step.
Allocate a fixed token budget per step, not per conversation.
If step 3 needs the output of step 1, pass the structured result only — not step 1's full chain-of-thought. A search step that returns ten results does not need to carry its internal ranking rationale forward into the synthesis step. What the synthesis step needs is the ranked list. Truncate aggressively at stage boundaries. This requires a deliberate design decision about what each step's output contract is. If the output is unstructured text, everything gets passed forward by default. If the output is a typed schema, only the schema fields travel between steps. The type boundary is what enforces the token budget.
Summarize and compact before handoff between pipeline stages.
The detailed reasoning from a retrieval step does not need to be present when the synthesis step runs. A structured summary does. Before passing control from one stage to the next, run a compaction step that takes the full output and produces a fixed-size digest. The digest carries the key facts forward. The full reasoning trace stays available for debugging but does not enter the production call chain.
The compaction step is itself a model call, which costs something. It is almost always worth it. A 3,000-token reasoning trace compacted to a 400-token structured summary saves 2,600 tokens on every subsequent step in the pipeline. On a ten-step pipeline running thousands of queries, the compaction cost is negligible against the savings.
Set a hard context limit per step and fail loudly when exceeded.
Silent context overflow produces subtly wrong outputs that reach users. A model operating with 2% of its context window available for the actual task will not return an error. It will return a plausible-sounding but degraded answer that passes surface-level quality checks. The failure mode is invisible unless you are actively measuring context size per step.
Adding a hard limit is straightforward: measure the token count before each model call and raise an exception if it exceeds a threshold. The threshold should be well below the model's maximum, not a fraction of the maximum. If the step budget is 2,000 tokens and the accumulated context is 8,000, the pipeline is not "within limits" — it is already broken.
Failing loudly means the engineering team sees the problem. Silent degradation means users see it first.
Three metrics catch context growth problems before they affect users:
These are not difficult to collect. A wrapper around the model call captures the prompt token count before sending. The difficulty is that most orchestration frameworks do not expose this by default. You have to add the instrumentation yourself.
The reason context growth catches teams off guard is that it does not look like a bug. The pipeline architecture that worked in a stateless test environment is also the architecture that fails under stateful production load. Nothing changed between test and production except the accumulated state.
The fix is to treat context as a resource with a budget, the same way you treat memory or API calls. Each step has an input budget and an output budget. The input budget caps what the step can receive. The output budget caps what the step passes forward. The pipeline enforces both.
An agentic pipeline that has no explicit context budget is running without a key operational constraint. It will work until it does not, and the failure signal will be subtle enough that users notice before dashboards do.
Context window management and instrumentation for production agentic systems are part of the consulting work I do at hannune.ai.