#
The Agent That Trapped Itself: How Context Eviction and Deduplication Broke on a Go Project
A deep dive into a real failing run in the Legion multi-agent hub: the anatomy of an infrastructure failure, fixing overly "smart" mechanisms, and an unexpected qualitative leap after shifting to an On-Demand architecture.
Anyone who has spent even a little time working with LLM-based agents knows that when an autonomous agent starts working on code, its context balloons with every tool call. It reads a file, runs grep, executes tests—and this entire history inevitably gets shipped along in the next prompt to the model. The cost of such a cycle grows almost quadratically, as the result of every previous step is paid for anew on every single iteration. But if the agent completes a certain part of a task during a session and switches to another question in conversation with the user, the "baggage" of the first phase remains as dead weight in the context, paid for by the user. Besides, why keep obvious things in memory that might have made it into the context via instructions, when an LLM is capable of recognizing most architectural patterns and "understanding" basic principles straight from its training data? That is precisely why, for the agentic system in Legion, I decided to implement context eviction mechanisms and cranked the eviction dial to "maximum" (though, to be honest, I hadn't planned to).
In the Legion multi-agent hub, a team of agents works on tasks within virtual "rooms" sharing a cloud repository. The idea of making the context as token-efficient as possible seemed flawless, but in September it led to a sobering case study: an agent methodically studied the codebase for a full hour... and didn't write a single line of code. It didn't hallucinate, nor was it lazy—it simply physically couldn't reach the contents of the files it had opened a second ago.
Below is the full investigation report, backed by fresh metrics and the results of qualitative growth in October.
#
Part 1: What We Built: Anatomy of a "Context Gateway"
In July, we designed a deterministic layer positioned between the tool execution loop and the model provider. It consisted of three main components:
Observations: The full result of every tool execution was saved in external storage, leaving behind only a lightweight reference in the context.
2.
Receipts: Outdated results in the request were replaced with a short stub. It contained the tool name, purpose, line range, hash, and the volume of evicted tokens (while keeping thetool_use /tool_result pairs structurally valid).
3.
Read Ledger: The system remembered which file ranges the agent had already received, and upon a repeated read of an unchanged fragment, it returned a receipt instead of text, saving on repeat costs.
The default policy (eager) left only the three most recent tool results "raw" within a limit of 6,000 tokens. Offline replay on 29 anonymized trajectories showed a rosy picture: total incoming traffic was reduced by 53%. But token metrics hid a fatal flaw.
#
Part 2: The Failure: One Hour of Work, Zero Lines of Code
On September 17, a backend agent was assigned a task: add a Kanban board endpoint to a Go backend (fetch tasks from a tracker, return ID, title, and stage). This was already a retry attempt (Retry 3). An hour later, the dispatcher forcefully terminated the session due to a time limit.
Logs showed 126 tool calls (99 of them bash, 18 observation reads, 9 read_file), but not a single code modification. The agent’s internal reasoning vividly illustrated its state:
"Let me look at the repository map output which got evicted" — it noticed the eviction almost immediately. #
"Hmm, this is odd — normally read_file returns content. The receipts indicate the tool result was evicted" — realizing a receipt was arriving instead of code, the agent tried to bypass the limitation viacat . #
"I already read it twice and both were evicted from context" — a classic infinite reading loop. #
"I've spent a LOT of budget. Let me be decisive" — panic was mounting, but it critically lacked fresh signatures to write code.
#
#
Part 3: The Hypothesis: Two Correct Mechanisms Creating a Deadlock
The investigation revealed a classic deadlock of two independent, yet inconsistent, logic flows:
"Always" (Eager) Eviction, Not On-Demand. The context assembler ruthlessly turned any file read three calls ago into a receipt, even when there was plenty of free space in the window (for instance, only 20K tokens out of 100K were occupied). During development, I chased context window optimization, losing sight of the fact that offline replays count savings, not successful task completion.
2.
Room-Level Deduplication Combined with Session-Level Retention. Observation artifacts were searched by the keyroom + tool + arguments + file fingerprint . As a result, the second and third attempts to solve the task found the cache of the first attempt, marked the read as a duplicate, and returned... a receipt again.
A vicious circle formed: The Context told the agent: "There is no text here, re-read it," while the Read Ledger cut it off: "You already got this, here is a stub." The agent fell back to cat, where aggressive eviction triggered again—and the file vanished once more. Meanwhile, the manifest was perceived by the model as a direct prohibition (Do NOT re-read).
#
#
Part 4: Corrective Actions: What Changed
Following the post-mortem, the gateway architecture underwent four key changes:
on_demand Eviction Mode Instead of eager. Data is now evicted only when the total request genuinely exceeds the budget limit, and strictly until normal bounds are restored. The old behavior is preserved as an optional mode for comparison.
2.
Content Retention Tied to Session. Caching observations across sessions to save space is retained, but the decision to*"serve text or a receipt"* is made strictly based on whether thecurrent session has seen this result (seen_in_session ). A new attempt starts with a blank slate.
3.
An Honest Manifest. The stub text was rewritten: it no longer forbids action, but explicitly states that there is no text in the context and the file is allowed to be re-read.
4.
Synchronization with Compaction. The assembler now explicitly informs the read ledger about which exact chunks of text were deleted during overflow, resetting the "read" status for erased fragments.
#
Part 5: Empirical Data: October Metrics and Qualitative Leap
Transitioning to the new architecture at the end of September allowed us to collect invaluable statistics. A comparison of average metrics between September and October reveals a radical shift in model working patterns.
- Prompt Caching (Cached / Uncached, million tokens)
Average cached: dropped from 6.48M to3.48M (-46%). #
Average uncached: dropped from 1.71M to1.03M (-40%). Interpretation: The absolute volume of the prompt dropped by almost half, while proportions remained intact. This proves the load reduction wasn't due to "cache magic," but because the agent simply required less context to solve tasks. In September, the model burned tons of tokens constantly re-reading and re-assembling lost data.
- Prompt / Reasoning / Completion Structure
Prompt (M): 8.01 →5.51 (-31%) #
Reasoning (K): 76.1 →58.2 (-24%) #
Completion (K): 62.2 →35.8 (-42%) #
Completion / Reasoning Ratio: 0.82 →0.65 (-20%) #
Reasoning / Prompt Ratio: 9.28 →11.2 (+21%) Key Metric Signals:
Collapse of Verbosity (Completion/Reasoning): In September, the final response volume was almost equal to the reasoning volume. Lacking facts in context, the model "improvised" code in the response to compensate for memory gaps. In October, agents began thinking deeply while answering concisely and to the point (-42% on completion text volume). #
Disappearance of "Despair Spikes": The September sample contained an anomalous case (15.4M prompt / 152K reasoning / 111K completion) reflecting a deep cyclic agent meltdown. In October, such extreme outliers vanished completely.
#
Part 6: Why Go Development Transformed Qualitatively
The ultimate proof of correctness manifested in practical software engineering.
The Go language is extremely strict, idiomatic, and devoid of dynamic magic: it relies on explicit structures, strict error handling (if err != nil), context.Context propagation, and rigorous configuration typing.
In September: Receipts erased low-level details. Trying to write a configuration or service, the agent lost the structure of production files and neighboring packages, rewrote configs "from scratch" while wiping ports and secrets, or generated foreign code alien to the project. #
In October: Retaining real code in context crankedIn-Context Learning (ICL) to the maximum. The agent shifted into precise mimicry mode: it sees real signatures, copies architectural layouts (handler -> service -> repository ), and neatly embeds new microservices into the existing monolith without breaking dependencies.
#
Key Takeaways for Agent System Developers
Evict on demand, not on a schedule. A fixed rule to "delete everything older than N steps" breaks workflows when plenty of free space remains in the window. 2. Isolate session contexts. A knowledge base and artifact cache can be shared across a project, but the decision of whether to show text to an agent must be made strictly at the level of its personal session. 3. Speak honestly to the model. Instructions like "do not re-read" paralyze the LLM. Phrasing it as*"the text is not in context, invoke the tool again if needed"* restores its freedom of action. 4. Token metrics do not equal success metrics. Offline optimization targeting input/output volume can quietly kill an agent's ability to solve practical problems. 5. Read the Reasoning. Phrases like*"both were evicted"* in logs signal a systemic failure long before a time-out triggers.
Trying to save a couple of hundred thousand input tokens using dubious stubs leads to wasting millions of tokens on endless debugging and hallucinations. The balance we found (On-Demand + honest receipts) elevated autonomous agents from hesitant juniors to reliable Middle+ engineers.