Every few weeks another framework ships a bigger context window and someone declares agent memory solved. It isn't. I've watched three separate production agents degrade in the exact same way — not because they forgot something important, but because they remembered too much and couldn't tell what mattered anymore. The fix wasn't more storage. It was an eviction policy.
Here's the hot take: retention is not the hard problem in agent memory. Forgetting is. Most teams building "memory" into their agents are really building append-only logs with a vector index bolted on for search. That's not memory, it's a diary nobody edits. And a diary that never gets edited eventually buries the one relevant entry under ten thousand irrelevant ones — at which point your retrieval step is doing archaeology, not reasoning.
The symptom is specific and recognizable if you've run an agent past a few hundred sessions: retrieval quality degrades, but the failure looks like a prompting problem, not a memory problem. The agent contradicts itself. It re-asks questions it already has answers to. It surfaces a fact that was true three iterations ago and has since been superseded, and states it with total confidence because nothing marked it stale. You go tune the retrieval prompt, add reranking, bump top_k
. None of it fixes the actual defect, which is that the memory store has no concept of a fact going bad.
The naive design is seductive because it requires zero decisions up front: every tool call, every user message, every intermediate reasoning step gets embedded and stored. Decisions get deferred to retrieval time, where a similarity search is supposed to sort it out. This works fine in demos because demos don't run long enough to accumulate contradictions.
Production agents do. Consider an agent that manages a user's project preferences over months. Session one: "I prefer TypeScript." Session forty: "Actually, switch this repo to Python." Both statements are now semantically close to any query about language preference — cosine similarity doesn't know which one is current. Worse, if the first statement was reinforced across more sessions (because it was true for longer), it will often outrank the correction in a naive top-k retrieval, because frequency and recency aren't the same signal and most memory layers only track one of them.
This isn't a retrieval-tuning problem. It's a data-modeling problem. You cannot rerank your way out of storing two contradictory facts with no relationship between them.
Treat agent memory like a cache, not a log. Concretely, that means three mechanisms most "memory" implementations skip entirely:
1. Explicit supersession. When a new fact contradicts a stored one on the same entity/attribute pair, don't just add the new fact — mark the old one as superseded and keep a pointer between them. This is cheap: a simple entity-attribute extraction pass on write ("user.language_preference: Python, supersedes fact_id 4471") turns your store from a bag of embeddings into something closer to a fact table with history. Retrieval then defaults to the current value and only surfaces history when explicitly asked.
2. Salience decay, not just recency decay. Recency-weighted retrieval (newer = higher score) is a start but conflates "recently said" with "currently relevant." A fact stated once in passing six months ago ("I'm allergic to shellfish") should outlive a fact restated weekly that's now irrelevant ("working on the Q2 report" — Q2 ended). Salience should be a function of how often the fact is retrieved and used downstream, not how often it was written. Facts nobody has retrieved in N sessions are candidates for archival — moved out of the hot index, not deleted, so you can still recover them if needed but they stop polluting top-k.
3. Write-time contradiction checks, not read-time cleanup. The temptation is to defer all of this to retrieval — over-fetch and let the LLM sort out contradictions in context. That works until your contradiction count outpaces your context budget, at which point you're paying token costs to have the model do data hygiene it's not well-suited for. Catching the contradiction at write time, when you already have the new fact and can do a targeted lookup against the same entity/attribute key, is orders of magnitude cheaper than re-deriving it from a pile of chunks at read time.
The counterargument deserves a real hearing, because it's not wrong so much as incomplete. With million-token context windows and cheap prompt caching, you genuinely can dump enormous amounts of raw history into context and let the model attend over it directly — no retrieval step, no embeddings, no eviction logic to build or maintain. For agents with short lifespans (single session, single task, bounded scope), this is the right call. Building a memory-eviction pipeline for an agent that only ever runs one conversation is solving a problem you don't have.
And there's a subtler point in favor of retention-heavy designs: sometimes the "irrelevant" old fact is exactly what you need for an edge case you didn't anticipate. Aggressive eviction risks discarding context that looks noisy in the 99% case but is load-bearing in the 1%. If you archive instead of delete, this risk is mitigated — but archiving has its own cost in system complexity.
Where this breaks down is exactly the scale at which context windows stop being an answer: long-running agents that accumulate state across weeks or months of interaction, where the token cost of re-sending full history every call becomes the dominant expense, and where contradiction — not volume — is the actual failure mode. At that scale, a bigger context window doesn't fix a stale fact ranking above its correction; it just gives the stale fact more company.
If you're building an agent that's meant to run for a single session, don't build a memory system — you don't need one, and a big context window is genuinely the simpler, correct answer. But if your agent is meant to accumulate knowledge about a user, a codebase, or a project over time, treat memory as a system with a write path, a supersession model, and a decay function — not a table you only ever append to. The teams shipping reliable long-running agents right now aren't the ones with the largest vector index. They're the ones whose memory layer knows when a fact has gone bad, and quietly stops surfacing it before anyone notices it was wrong.