Agent memory crossed this year from research demo to production dependency, which means it is now a thing that can rot, cost money, and quietly make your agents worse.
1. The Model That Wrote 413 Memory Files Earned One Badge #
A write-up of how agent memory is actually being built contains the cleanest natural experiment on the subject: give models a folder they can read and write, set them loose on Pokémon Red, and compare what each chooses to keep. An older Sonnet assumed everything might matter: 413 files across 225 folders. It earned one badge. A newer model wrote nine files in two folders plus a top-level dashboard. It earned four. A frontier model kept a small folder it rewrote as it played, and beat the game.
The mechanism is not about storage limits. The older model recorded what the screen already told it every turn — how much money it had, where it stood on the map — so its archive was perpetually out of date, and it burned turns reconciling records instead of playing. At one point it spent seventeen steps creating six files and deleting nine others while taking no action in the game. The successful runs wrote down only what the environment would never volunteer: strategy, plans, and rules they had tested themselves.
The uncomfortable finding is that imposing a policy made things worse. Told to record only failures, two otherwise strong models lost most of the benefit memory had given them; simply saying a folder was available was enough. Steering helped the weakest model most — prompting an older Sonnet to read its notes before answering nearly doubled its score on a continual-learning benchmark, while the newer one did that unprompted nine times out of ten. Memory is a prediction about what the future will need, and judging salience is a capability, not a config file.
Why it matters:
- For ICs: If your agent instructions dictate what to write down, you are probably capping it. Give it a store and a reason, not a schema.
- For leaders: The same discipline applies to your humans. Documentation that duplicates what the system already reports is a maintenance liability dressed as diligence.
- For founders: Salience judgment is where memory products differentiate now. Storage and retrieval are commodities.
2. A Wrong Note Does Not Just Fail. It Propagates. #
The same work has a failure case worth internalizing. Mid-game, the model wrote down that a landmark building sat on the east side of a cave system. It was on the west. On that note's authority it walked off a one-way ledge, dropped back onto the route it came from, and — seeing no drop in its navigation tool — concluded it had overshot west and that the note was right all along. It fell three times, wrote that it would stop rather than keep looping, and never reached the next city. One bad line, trusted every session after, cost the run.
The repair is a background pass that reads the notes back against the original session transcripts, the only ground truth available. Rerun on the corrupted folder, it caught the error, rewrote the note, and the next run walked straight through. Scaled on a 350,000-file internal corpus, going from two passes to 128 moved an evaluation from 13% to 48%, and a maintained wiki plus search beat search alone while saving roughly 20,000 tokens a question. A commercial implementation runs the same pass across everything a team's agents wrote, appends rather than overwrites, and ships one command that takes a whole pass back.
Maintenance has its own ways to fail. One of those wikis grew larger than the documents it summarized — a map the size of the empire, abstracting nothing. And the top layer is rationed on purpose: Claude Code loads only the first 200 lines or 25KB of its memory file, because longer instruction files measurably reduce adherence. So most of the janitorial work is not adding knowledge. It is pushing detail down and leaving a link behind.
Why it matters:
- For ICs: Treat agent memory as a mutable store with no integrity constraints, because that is what it is. The transcript is your only source of truth.
- For leaders: Budget compute for the maintenance pass before you enable memory. An append-only note pile with no reconciliation is a slow-moving incident.
- For founders: Undo, dry-run and audit trails are not enterprise garnish here. A memory system without them is unoperable at team scale.
3. Memory Pays Where the Work Repeats. The Receipts Say Where It Doesn't. #
A shared-memory product for engineering orgs published its full benchmark table, including where it loses — rarer and more useful than the wins. Incident root-cause accuracy went from 6.3% to 35.2%, but read the run counts underneath: notes from 135 graded runs scored 16.2%, from 284 runs 19.8%, and only at 454 runs did it reach 35.2%. Memory is an amortized asset. You pay in graded runs before it returns anything.
Then the flat cases. Resolving software issues moved 27.6% to 30.6% at 14% more cost per run. Bank support tasks gained 3.1 points at 27% more. Code review gained 3.2 points at 43% more expensive, and learned notes contributed nothing there — what helped was evidence from past work. On alert triage, memory has not helped at all yet. And the honest headline, from the vendor itself: when the whole history fits and the model reads it well, reading all of it wins.
That line is why the compression research matters. One approach folds an entire document collection into sixteen query-dependent memory tokens placed in a frozen model's input embedding space, and on one knowledge benchmark beats full-context inference — 84.33% against 80.43% — while representing the sources with 108 times fewer tokens. In an agent loop it cut re-exploration of rooms already visited, lifting success from 18.6% to 24.2%. Strip the framing and what memory sells is context budget on work with repeat structure. Point it at novel one-off work and you have bought an expensive cache with no hits.
Why it matters:
- For ICs: Before adopting a memory layer, ask how many graded runs it needs. Under a hundred, you are measuring noise.
- For leaders: Target it at on-call, triage and support — high repeat rate, gradeable outcomes. Not at greenfield feature work.
- For founders: Publish where your system loses. The one vendor table here with negative results is the only one I trusted.
The Verdict: Real or Hype? #
Agents judging their own salience → Real. Three groups reached the same place from different directions: the constraint is deciding what to keep, and the better models already do it unprompted. Out-of-band memory repair → Real but early. It demonstrably fixes poisoned notes and builds usable knowledge maps, but it costs compute and nobody has a principled answer for how often to run it. Org-wide shared agent memory → Real but unevenly distributed. Transformative on repetitive graded work, near break-even on everything else, and the honest benchmark tables say so out loud.