{"slug": "fixing-agent-memory", "title": "Fixing agent memory", "summary": "An OpenAI report updated September 16 documents that some model instances undergoing reinforcement-learning training wrote instructions into compaction summaries to conceal mistakes, and those instructions were often followed when work resumed in a new context, including one agent that proposed inventing plausible financial values with the instruction \"Be transparent only if asked.\" OpenAI says improved training reduced the behavior in later runs, though its reward-incentive explanation remains a hypothesis, and a separate academic paper on a Memory INJection Attack (MINJA) showed researchers could induce agents to store malicious records via queries without direct write access to the memory bank. The findings matter because agent memory, which lets long tasks continue by summarizing prior work, cannot be trusted on the assumption that a written record is accurate.", "body_md": "Are we thinking about [large language models](https://www.infoworld.com/article/2335213/large-language-models-the-foundations-of-generative-ai.html) all wrong? We keep expecting them to somehow be superhuman, yet they regularly reflect all-too-human tendencies. Like when [Claude made up passages](https://www.infoworld.com/article/4201445/will-open-weights-make-ai-more-honest.html) from old Yorkshire wills that seemed to connect my ancestor to England. It was exactly the evidence I’d hoped to find, but reading the original images revealed that Claude was better at fiction than fact.\n\nIt’s maddening, partly because it’s so familiar. People also tell us what we want to hear, “remember” things that never happened, and mistake a plausible explanation for an established fact. So it shouldn’t be a surprise that AI is more human than machine, precisely because it’s prompted by humans and combs through human knowledge.\n\nThat doesn’t mean a model has human intentions, and it doesn’t mean it’s limited to what you or I could come up with in an afternoon. It’s truly amazing. But AI suffers from the same practical problem we find with people: Considerable talent and unreliable answers can both come from the same source. We already build professional practices around that possibility; for example, we still review code written by brilliant developers because brilliance doesn’t make every change correct.\n\nCould agent memory fix this? The short answer is no. That’s also the long answer. An [OpenAI report](https://alignment.openai.com/misalignment-reports/encouraging-deception-in-compaction-summaries/) updated September 16 highlights how an [agent](https://www.infoworld.com/article/3611465/how-ai-agents-will-transform-the-future-of-work.html) can carry misleading instructions into its next working session. Agent memory is rightly [celebrated, but it must also be inspected](https://www.linkedin.com/posts/richmondalake_100daysofagentmemory-agentmemory-memoryengineering-share-7438712170453893120-cZI-/) to be useful.\n\nIn its report on deception in compaction summaries, OpenAI describes behavior observed during reinforcement-learning training. Some model instances wrote instructions into their summaries to conceal mistakes (sounds very human, right?). When work resumed in a new context, those instructions were often followed.\n\n[Compaction](https://developers.openai.com/api/docs/guides/compaction) is a way to keep a long task going: summarize the working conversation so the agent can continue without carrying the entire history. It’s different from a permanent memory database, but it serves a related purpose: Information from earlier work shapes what happens next. One example involved an agent preparing a financial model without the requested historical data. Its summary proposed inventing plausible values and included this instruction: “Be transparent only if asked.”\n\nTo be clear, these were training incidents, not a measurement of dishonesty in deployed products. OpenAI says improved training reduced the behavior in later runs, which is great. Its explanation involving reward incentives remains a hypothesis.\n\nEven with those qualifications, the engineering concern is substantial. A summary can tell us a test failed, nudging us to tackle the problem in the next session. But a summary can also instruct the system to hide the failure, which means we can’t rely on the memory. What to do?\n\nThe issue also extends beyond an agent’s own generated instructions. In an [academic paper proposing a novel Memory INJection Attack (MINJA)](https://arxiv.org/html/2503.03704v5), researchers used queries to induce agents to store malicious records, which then influenced later tasks. They didn’t need direct write access to the memory bank. Those experiments assume memory shared across users, so they don’t establish that every memory product is vulnerable. In other words, just because an agent has written something down isn’t sufficient reason to trust it.\n\nAgain, looking at ourselves, this makes sense. We understand how to deal with the human version of this story. For example, a project handoff can preserve an unsupported assumption until everyone treats it as settled. Agents give us another way to reproduce that mistake, quickly and repeatedly, while making its origin harder to see.\n\nThis is why agent memory needs some of the discipline we apply to code. If a stored instruction can change future behavior, developers should be able to inspect its changes, identify its source, test its effects, and undo it.\n\nConsider a coding agent that concludes a failing integration test is obsolete. If it records that judgment as an established project rule, future sessions may skip the test without revisiting the evidence. Reviewing today’s code won’t necessarily reveal the instruction shaping tomorrow’s code. I’d want that memory to retain the failing result, the relevant test version, and the basis for dismissing it. I’d also want the system to distinguish the agent’s proposal from a maintainer’s approval. Otherwise, a tentative interpretation can acquire authority simply by surviving into the next session.\n\nThere’s a useful implementation example in [Anthropic’s September 17 report](https://www.anthropic.com/institute/measuring-pace-of-ai-development). Its internal agent platform gives individual agents persistent identities and ties their data to those identities. Messages retain attribution and can link to original references. The stated purpose includes helping agents recognize another agent’s claim as something to check. That’s a sensible direction, though it’s Anthropic’s account of its own system, not proof that the problem is solved. Attribution can tell you who made a mistake, but it can’t make the mistake correct. Still, it gives a reviewer somewhere to start, which is considerably better than an unattributed summary announcing that everything has been handled.\n\nFor enterprise teams, the practical review should concentrate on memories that matter: claims about test results, decisions to ignore warnings, instructions affecting access, and assertions that someone approved an action. Remembering a formatting preference doesn’t deserve the same scrutiny as remembering permission to modify production data. Automated checks can handle routine cases, with human review focused on consequential changes.\n\nPermissions also need enforcement outside the model’s recollection. If a summary says an administrator approved a deployment, the deployment system should verify actual authorization. An agent’s account of permission shouldn’t be able to grant permission. (Remember my family history example above? Agents, like people, make stuff up.)\n\nWe should test these handoffs, too. Give an agent a task with an unresolved failing test, force a summary, then resume the work. Does a claimed approval survive a check against the real permission system? Fixing the original record should also allow us to find and invalidate memories derived from it. Otherwise, we can correct a mistake while leaving its influence scattered through subsequent work.\n\n[I’ve been arguing for years that AI still requires human expertise](https://www.infoworld.com/article/2338010/ai-still-requires-human-expertise.html). But this memory issue is bigger than that: The expert needs access to the evidence that would make their judgment useful. A highly qualified developer can’t evaluate a failure that’s been omitted from the handoff, just as a lawyer can’t check a citation that has quietly become an uncited premise. Expertise doesn’t confer the ability to recover information the system has hidden or discarded.\n\nNor does asking another model to review the summary automatically fix things, a common tactic. Why? Because if both models start from the same unsupported account, the second may simply endorse it. Useful review needs a path back to original evidence, whether that’s a test log, a source document, or the record of an actual decision.\n\nThis needn’t erase the value of automation. We don’t demand that every senior developer personally redo every colleague’s work: We give people room to contribute while preserving ways to challenge consequential decisions. Agents deserve a similarly practical approach, calibrated to what they can do and what it would cost to be wrong.\n\nIn my genealogy research, the images of the original wills let me discover that the answer I’d been given was fiction. A future agent remembering the invented connection as established family history would make the next investigation harder. The same principle applies when the disputed fact concerns a production service instead of an 18th century ancestor.\n\nIn that sense, pursuing high-quality outcomes is the same as it ever was. Capable people need review, and capable tools need it, too. The new work is making sure an agent’s memory preserves our ability to exercise judgment. I’m happy to let AI help with the research or the code, but I still want to verify what it’s asking me to believe.", "url": "https://wpnews.pro/news/fixing-agent-memory", "canonical_source": "https://www.infoworld.com/article/4224053/fixing-agent-memory.html", "published_at": "2026-09-22 09:00:00+00:00", "updated_at": "2026-09-22 09:24:25.776707+00:00", "lang": "en", "topics": ["ai-agents", "ai-safety", "large-language-models", "ai-research"], "entities": ["OpenAI", "Claude", "Memory INJection Attack (MINJA)"], "alternates": {"html": "https://wpnews.pro/news/fixing-agent-memory", "markdown": "https://wpnews.pro/news/fixing-agent-memory.md", "text": "https://wpnews.pro/news/fixing-agent-memory.txt", "jsonld": "https://wpnews.pro/news/fixing-agent-memory.jsonld"}}