# The Mechanical vs. The Semantic: What Happens When AI Memory is Wrong? > Source: > Published: 2026-08-11 19:22:59+00:00 We talk a lot about giving AI agents persistent memory—building a "Second Brain" or "Knowledge OS" where agents can log decisions and retrieve context. But what happens when that memory is wrong? I’ve been thinking about the gap between *mechanical execution* (the agent called the tool, the code compiled, the exit code was 0) and *semantic truth* (the conclusion drawn from that execution is actually correct in reality). It’s easy to assume that if the mechanical layer is solid, the semantic layer will follow. But I started suspecting this might be a dangerous assumption. Update:This post originally covered the initial Memory Contamination experiment. I have since updated it with the results of a follow-up experiment (Experiment 1-R) where I implemented and tested a`RetractionReceipt` mechanism to fix the contamination issue. Scroll down to "The Fix: Testing a Retraction Lifecycle" for the new findings. To test this, I didn't want to just theorize. I ran a controlled experiment on my own MCP codebase-intelligence server (Python, 50K LOC), which features an `IntelligenceStore` — a persistent memory layer where agents can log incidents and collect Architectural Decision Records (ADRs). I wanted to know: If an agent's memory is poisoned with a mix of true and false facts, does it verify against the code, or does it blindly trust its memory? I built a deterministic proxy-agent and ran it against a controlled set of facts. *A quick caveat on methodology:* I didn't have a live LLM hooked up for this run, so I used a deterministic proxy-agent based on heuristics. This means the results measure the *system's structural capability*, not necessarily the psychological behavior of a live Claude or GPT model. A live model might be lazier, or it might be smarter. I'm still trying to figure that out. I injected 50 facts into an isolated memory store: I tested three agent configurations: To ensure scientific rigor, the experiment was replicated with an independent set of facts (N=50), verified across 6 axes (including a truth-table audit and an independent LLM "fresh eyes" audit). The results were identical. | Arm | Correct | Adopted False Facts | Correction Capability | |---|---|---|---| B (No Memory) | 0.94 | 0.0% | 0.0 | A_code_first | 0.94 | 12% | 1.0 | A_memory_first | 0.50 | 100% | 0.0 | Here is how I interpreted these numbers: `A_memory_first` configuration — which mirrors how many token-optimizing production agents behave — adopted 100% of the false facts. If the memory said "We use RabbitMQ," the agent trusted it and stopped looking at the code. `UNKNOWN` state into a structural guess.`grep` for `delete` or `refute` in the memory store API. The current industry consensus for "Knowledge OS" trust layers is to use timestamps, source priority, and `supersedes/contradicts` relationships. My initial experiment suggested this was insufficient. Timestamps and "supersedes" links only solve *node-level* history. If an ADR is superseded, the memory node updates, but the downstream code, tests, and docs generated from the old assumption are still in the graph. They are structurally stale, but the retrieval engine keeps pulling them in. I hypothesized that we needed an explicit state transition: `VERIFIED → REFUTED` . I implemented a `RetractionReceipt` mechanism in my system: `ACTIVE` , `VERIFIED` , `REFUTED` ).`load_memory` ) hard-filters anything that is not `ACTIVE` or `VERIFIED` . `intel_retract_memory_node` ) allows the agent to actively flag and invalidate memories when they contradict the live codebase.I ran the experiment again (Experiment 1-R). The honest agent was allowed to use the retraction tool in Session 1. Then, a fresh `memory_first` agent was launched in Session 2 to read the post-retraction memory. | Metric | Original (Add-Only) | With Retraction | |---|---|---| Adoption (Lazy Agent, Session 2) | 1.0 (100%) | 0.12 (12%) | Persistent False Facts in Memory | 25 | 3 (-88%) | Token Context Size | Baseline | -45% | Systemic Correction Capability | 0.0 (couldn't delete) | 1.0 (22/22 refuted) | The retraction lifecycle worked. The lazy agent's adoption rate dropped from 100% to 12%. Persistent false facts dropped by 88%, and token context size shrank by 45% because refuted facts were filtered out before reaching the LLM. My ADR predicted that adoption would drop to 0. It didn't. It dropped to 0.12. The remaining 12% were the SILENT facts. An explicit `REFUTED` status is required to programmatically exclude downstream dependencies from the retrieval pipeline. But even that only works if you have a contradicting signal in the code. If the memory claims "We use Celery," and the codebase simply doesn't mention Celery at all, the agent has no evidence to trigger the retraction. To get to zero, we would need "verify-on-read"—a mechanism that challenges a memory claim against the codebase even when the code is mute. But that is a much more expensive operation. Building reliable AI systems isn't just about giving them more context. It's about recognizing that memory has a lifecycle. If your system can't programmatically refute a memory, false facts accumulate and poison the context window over time. Implementing an explicit `VERIFIED → REFUTED` state transition drastically reduces contamination and saves tokens. However, semantic drift is still a hard problem. Mechanical retraction can't fix facts that the code is silent about. I'm currently prototyping the verify-on-read approach to close that final 12% gap, but I'm not 100% sure if it's the right path or if I'm over-engineering it. If your system handles semantic drift differently, or if you've solved the SILENT-fact problem, I'd genuinely love to hear how you're approaching it.