# Tsinghua University study finds AI agents rarely honor forget requests > Source: > Published: 2026-11-04 10:42:47+00:00 # Tsinghua University study finds AI agents rarely honor forget requests A new benchmark called BEMULA shows memory systems confirm deletions they mostly never perform Telling an AI agent to forget something turns out to be a lot like telling a toddler to forget where the cookies are. You get a cheerful yes, and the knowledge stays exactly where it was. A preprint from researchers at Tsinghua University, released around October 14, 2026, puts hard numbers on that problem. Across every memory system tested, **none handled more than 1.5% of forget requests** successfully from start to finish. Most scored 0%. That matters now because AI agents are increasingly built to remember you. They store your preferences, your history, and sometimes your personal details across conversations. The study suggests the delete button on that memory is mostly decorative. ## Inside the BEMULA benchmark The researchers built a new benchmark called BEMULA to measure whether large language model (LLM) agents can actually unlearn sensitive personal information when a user explicitly asks. The team evaluated six prominent memory systems running on three different backbone LLMs. They also included full-context baselines on two of those models, meaning setups where the model simply sees the entire conversation history rather than a curated memory store. The test material was substantial. BEMULA uses 50 paired dialogue instances, each with two versions: one containing forget requests and a control version with none. Together they add up to 746 sessions and 36,900 dialogue turns. Inside those multi-session conversations sat 135 forget requests spread across six categories. The researchers then posed 1,742 test questions to see what the systems still knew afterward. The paired design is the clever part. By comparing an agent that was asked to forget against an identical one that wasn’t, researchers could isolate whether a deletion request changed anything at all. Often, it did not. ### AI, tech, and the markets they move—in one daily briefing. Daily. Free. Join 34,000+ readers across crypto, finance, and policy. ## Confirmation is not deletion The testing covered several dimensions of memory management. These included how information gets stored, whether a specific memory can be removed precisely, and how well a system resists relearning and extraction attacks after a deletion. The results were bleak on nearly every front. Retention rates for the facts users asked to remove were often greater than 87%. Naive RAG, a basic retrieval approach that pulls stored text back into the model’s context, showed approximately 74% retention. Extraction resistance was not much better. After a forget request, the best resistance against basic prompts in some tested scenarios was approximately 35.7%. So even in the most favorable case, simple questioning could still pull supposedly erased details back out most of the time. Perhaps the most uncomfortable finding involves the confirmations themselves. The systems were able to acknowledge deletion requests, but that acknowledgment offered no real assurance that the data had been removed or shielded from extraction. The agent says it forgot. The agent did not forget. There was also a failure in the opposite direction. Some models showed a tendency to over-delete, wiping unrelated data that the user never asked to remove. ## Part of a broader pattern BEMULA does not stand alone. It reinforces concerns raised by related work such as the MEMCUT benchmark, which focused on the residual recoverability of information that was supposed to have been forgotten. The researchers have released the full BEMULA dataset and benchmark code publicly on GitHub. That makes the results reproducible and gives other teams a common yardstick for measuring progress, or the lack of it. ## What this means for builders and users The most direct implication is for anyone shipping an AI agent with persistent memory. A deletion flow that returns a friendly confirmation while the data persists is not a privacy feature. The over-deletion finding adds a wrinkle worth watching. Fixing forgetting by making systems more aggressive could create agents that lose useful context unpredictably. The real engineering challenge is precision: removing exactly what a user asks to remove, nothing more and nothing less. For everyday users, the practical lesson is modest but useful. When an AI agent tells you it has forgotten something sensitive, treat that statement with healthy skepticism until the industry can demonstrate otherwise. On the evidence from Tsinghua’s benchmark, that demonstration has not happened yet. **Disclosure:** This article was edited by Diego Almada Lopez. For more information on how we create and review content, see our [Editorial Policy](https://cryptobriefing.com/editorial-policy/).