A coding agent that correctly diagnoses a failing test at turn three will sometimes, by turn forty, suggest reverting the fix it already applied. It has read the same log file five times, inherited every tool result along the way, and none of that volume made it more reliable. It made it worse. That’s context rot, and it isn’t really a memory failure. It’s what happens when a transformer’s attention gets stretched thinner than the problem needs [1].
In this article:
Context engineering is a different discipline from prompt engineering, even though people use the terms interchangeably. Prompt engineering is about writing the instructions well, the kind of phrasing and structuring work covered in an earlier post on prompt engineering best practices. Context engineering is about curating everything else a model sees at inference time: the system prompt, the tool definitions, the message history, and every tool result that has piled up along the way [1].
The mechanical reason bigger windows don’t fix this on their own comes down to attention. Transformers compute pairwise relationships between every pair of tokens in the input, so as the window fills, the model’s attention has to stretch across an n-squared relationship instead of a linear one. Anthropic’s own engineering team describes the resulting effect plainly: as the token count in context grows, a model’s ability to accurately recall specific information from that context goes down, even when the raw capacity to hold the tokens is technically still there [1].
That’s the part worth sitting with. A context window and a model’s ability to use everything inside it are two different properties, and a system built assuming they’re the same one will fail quietly, not loudly.
Position matters as much as volume. A 2023 study from Stanford and collaborators, published in the Transactions of the Association for Computational Linguistics, tested how well language models retrieve an answer from a long document depending on where in the document that answer sits [2]. The result is a U-shaped curve: performance is strongest when the relevant information sits at the very beginning or the very end of the input, and drops sharply when it’s buried in the middle, a pattern that held even for models explicitly built to handle long contexts [2].
This is why simply lengthening a system prompt or dumping more retrieved documents into a request doesn’t reliably help. If the one fact that matters lands in the middle third of a 50,000-token input, the odds it gets used correctly are worse than if it had landed at the top or bottom, independent of whether the model’s advertised context window could technically hold the whole thing [2].
Part of what makes context rot sneak up on people is that more counts against the window than the visible conversation. Claude’s own documentation is explicit that everything in a request counts toward the limit: the system prompt, every message including tool results, the tool definitions themselves whether or not a tool ever gets called, and the model’s own output from the previous turn, including any extended thinking [3].
Tool results are the quiet offender here. The same structured tool-calling mechanism that made function calling a foundation of agent design, covered in an earlier piece on the OpenAI function calling feature, is also what floods a session with raw output. A single tool call that returns forty lines of JSON gets added to the transcript in full, and it stays there on every subsequent turn unless something actively removes it.
Claude’s current lineup spans two window sizes: several current models, including Claude Sonnet 5, offer a 1M-token context window, while others, including Claude Sonnet 4.5, top out at 200k tokens [3]. Anthropic’s documentation names server-side compaction as the primary strategy for keeping long-running, agentic conversations from drowning in their own history once a session runs long enough for that gap to matter [3].
Anthropic’s guidance treats context management as an active, ongoing discipline rather than a one-time setup step, and it names four specific techniques worth understanding on their own terms.
Compaction summarizes older parts of a conversation once it grows long, preserving the decisions and details that matter while discarding redundant tool output, so a session can keep going with only modest performance loss instead of dragging every prior tool result forward indefinitely [1].
Structured note-taking has an agent maintain its own external memory, something like a persistent NOTES.md file, and pull specific notes back in only when a context reset actually requires them, rather than keeping the full history live at all times [1].
Just-in-time retrieval flips the usual instinct to pre-load everything a task might need. Instead, an agent keeps lightweight references, like file paths or record identifiers, and only pulls the full content in at the moment it’s actually needed. That’s a meaningfully different design philosophy from stuffing a context window with every retrieved document up front, the approach a lot of early retrieval-augmented systems leaned on before context windows and retrieval pipelines matured into the more selective approach covered in The RAG Revolution [1].
Sub-agent isolation delegates a narrow piece of work to a separate agent that does its own reading and reasoning in its own window, then returns a condensed result. Anthropic’s own framing puts that returned summary at roughly one to two thousand tokens, regardless of how much the sub-agent had to read to produce it [1].
Anthropic’s engineering team puts the underlying philosophy in a single sentence: “Good context engineering means finding the smallest possible set of high-signal tokens that maximize the likelihood of some desired outcome” [1]. That’s worth sitting with because it reframes the whole problem. The instinct when an agent underperforms is usually to add more: more instructions, more retrieved documents, more prior turns kept alive in the transcript. The better first move is closer to subtraction, deciding what can be summarized, deferred, or delegated out of the window entirely.
None of this requires exotic tooling to start applying. Before shipping a long-running agent or a multi-turn workflow, four checks catch most of the problems above. First, look at what’s actually accumulating in the transcript across turns, specifically raw tool output, and decide whether any of it should be summarized or dropped rather than carried forward untouched. Second, if a task depends on one specific fact buried inside a long document or a long system prompt, don’t assume the model will find it just because the window is technically big enough; move that fact toward the beginning or end if possible. Third, treat tool definitions as part of the budget too, since they’re counted whether or not they’re ever invoked, and a tool list that’s grown organically over months is worth auditing on its own. Fourth, for anything that runs long enough to risk drift, decide in advance whether compaction, external notes, or sub-agent delegation is the right mechanism, rather than discovering the answer only after a session degrades in production.
This is also, notably, no longer just a case of good practice with no formal name attached. An independent, unofficial guide to the Claude Certified Architect, Foundations exam lists Context Management & Reliability as one of the five graded domains, at a reported 15% weighting, covering context window management, caching strategies, and reliability in long conversations, though that specific percentage comes from a third-party study resource rather than Anthropic’s own exam guide PDF, which wasn’t independently verifiable through this piece’s research [4][5]. The certification’s existence and its domain structure are confirmed on Anthropic Academy’s own page, a detail worth mentioning alongside my own experience sitting that exam earlier this month, covered in Back to Writing: My Claude Certification Week [4].
Whether or not a formal exam ever grades it, the underlying discipline is the same: a bigger context window buys capacity, not attention, and the systems that hold up under real use are the ones that treat every token entering that window as something that has to earn its place.
If you enjoyed the article and wish to show your support, make sure to: Context Engineering: Drift, Bloat, and Lost Attention was originally published in Towards AI on Medium, where people are continuing the conversation by highlighting and responding to this story.