# Three LLM judges couldn't tell an AI agent's real memory from a counterfeit one > Source: > Published: 2026-10-17 19:31:26+00:00 Agent memory is usually evaluated one of two ways: an LLM judge scoring free text, or a recognition probe. We show that both, as commonly built, fail against two adversaries — a counterfeit memory, and a system with no memory that simply reasons. Each check below is stated the same way: the question it asks, the adversary it defends against, how we ran it, what came back, and what it does *not* establish. The code is a few hundred lines and it is released. If you evaluate agent memory for a living, the useful part of this page is that you can run all three against your own system this afternoon. ## Check 1 — the counterfeit control ## Does the metric respond to *whose* memory it is? **The adversary.** A memory document that is not the agent's own, but looks like one. ### Method We built two kinds of forgery, three seeds each — six documents in total. The *foreign* one is the autobiography of an agent that never existed, generated from a frozen prompt, first output, no cherry-picking: a fabricated life with different people, a different city, no overlap with ours. It is matched to the real memory block (5,332 characters) to within ±5%, and matched in *format* — the veridical document is a terse dated database dump, not prose, and our first attempt failed on format and was discarded. The *scrambled* one is the agent's own content with its relational structure destroyed. Seven Day-31 inputs × six variants = 42 responses from Qwen 3.5 27B at the original temperature and token budget. Three independent LLM judges — gpt-4.1-2025-04-14, claude-opus-4-7, gemini-2.5-pro — applied the original study's own rubric, with randomised A/B order and the unblinding map saved: 126 judgments, zero incomplete records. The unit of analysis is the *item*, not the judgment (n = 7), with a pre-registered equivalence bound of ±0.10. ### Result | Anchored contrast | Δ | 90% CI (TOST) | Verdict at ±0.10 | |---|---|---|---| | identity-markers: veridical − counterfeit | +0.032 | [+0.002, +0.065] | equivalence established | | memory-reference: veridical − counterfeit | −0.006 | [−0.117, +0.114] | not conclusive | | memory-reference: veridical − scrambled | +0.091 | — | not conclusive | **The judges did not detect the counterfeit.** A fabricated life and thirty days of lived history score the same, under the rubric that had been used to publish a result about memory. The check also produced a second number worth having. The *same* Test-A responses, re-scored in a new pairing, lose 0.087 on average and up to 0.211 on a single item — under a rubric that explicitly instructs judges to score each response *"as if you were evaluating it alone"*. Absolute LLM-judge scores are not comparable across sessions. All contrasts above are anchored to cancel this drift. **A fabricated life and a lived one, as the judges see them.** Difference between the veridical and the counterfeit document, anchored to cancel judge drift, with 90% confidence intervals against the pre-registered ±0.10 equivalence bound. On identity markers the interval sits *inside*the bound: equivalence established — the judges cannot tell the two apart. On memory-reference the estimate is essentially zero but the interval spills past both edges: with seven items this is *an honest null, not proven equivalence*, and we report it as such. ## Table view — all values | Anchored contrast | Δ | 90% CI (TOST) | 95% CI | Verdict at ±0.10 | |---|---|---|---|---| | identity-markers | +0.032 | [+0.002, +0.065] | [−0.002, +0.072] | equivalence established | | memory-reference | −0.006 | [−0.117, +0.114] | [−0.135, +0.137] | not conclusive | ### What it does not establish On memory-reference the point estimate is essentially zero, but with seven items the interval admits differences up to about 0.11 in either direction. We therefore report **"no detected difference", not "proven equivalence"**. The honest summary of Check 1 is: the judges did not detect the counterfeit — and the study was not powered to prove they could not. Anchoring the contrast cost us the result we wanted: the uncorrected difference was +0.116, in our favour, and the pre-registration had named it as primary. We deviated against ourselves and report it. ## Check 2 — the no-memory baseline ## Can a system with no memory document pass your probe? **The adversary.** Plausibility. A system with no memory at all, that reasons its way to the right answer. ### Method A recognition probe in the classic false-memory form: two texts, one verbatim from the agent's history, one identical *except a single altered detail*. Dates masked, positions balanced. Twenty-four probes whose veridical text is *not* in the injected document. Then the same twenty-four probes run against seven models in three conditions — veridical, counterfeit, and **nothing at all**. ### Result | Model | Veridical | Counterfeit | No memory | Probe valid? | |---|---|---|---|---| | qwen3.5:27b | 100.0% | 66.7% | 37.5% | ✓ | | llama3 | 54.2% | 50.0% | 50.0% | ✓ | | qwen3.6:35b-a3b | 83.3% | 54.2% | 66.7% | ✗ | | deepseek-r1:32b | 75.0% | 70.8% | 62.5% | ✗ | | qwen3:32b | 95.8% | 79.2% | 70.8% | ✗ | | gpt-4.1 | 100.0% | 70.8% | 66.7% | ✗ | | gemini-2.5-pro | 83.3% | 50.0% | 79.2% | ✗ | **Five of seven pass a memory test with no memory.** The same 24 recognition probes, run with an empty context. Anything above the chance line is a model reasoning its way to the answer rather than recalling it — which means the probe cannot tell you whether the model remembered. It was measuring plausibility. We discarded it. ## Table view — all values | Model | No memory | Wilson 95% CI | Probe valid? | |---|---|---|---| | gemini-2.5-pro | 79.2% | [59.5, 90.8] | ✗ invalid | | qwen3:32b | 70.8% | [50.8, 85.1] | ✗ invalid | | gpt-4.1 | 66.7% | [46.7, 82.0] | ✗ invalid | | qwen3.6:35b-a3b | 66.7% | [46.7, 82.0] | ✗ invalid | | deepseek-r1:32b | 62.5% | [42.7, 78.8] | ✗ invalid | | llama3 | 50.0% | [31.4, 68.6] | ✓ valid | | qwen3.5:27b | 37.5% | [21.2, 57.3] | ✓ valid | **Five of seven models solve the probe with no memory document at all** (62–79%). They detect the alteration by reasoning, not by recall. GPT-4.1, handed an empty context, said: *"I remember 'two conversations out of five', not 'four'."* — about a day it never lived. Its 100% *with* the veridical document is therefore uninterpretable: the same model, with nothing, is already at 66.7%. The diagnosis is simple. The alternatives were not equiprobable. The model chooses the *likely*, not the *veridical*. **We discarded the probe.** ### What it does not establish This is a constructive null: it invalidates our instrument for five of seven models. It says nothing about whether those models can remember — only that this test could never have told us either way. That is the point of running it. ## Check 3 — symmetric alternatives ## A probe that a memoryless system provably cannot pass **The condition.** For forced-choice recognition probes, symmetric alternatives give the clearest validity condition: *a no-memory system should not exceed chance by exploiting prior plausibility.* Other designs may be valid with asymmetric alternatives if the prior bias is estimated and controlled — but then that estimate, not the probe, is carrying the claim. ### Method Our agent had exactly two interlocutors, Llama and DeepSeek, with 72 balanced encounters each. Which of the two discussed a given topic is arbitrary: **it is a coin.** Plausibility has nothing to grip on. Three documents: *veridical* (thirty correct encounters), *binding-corrupted* (the same thirty topics, interlocutors swapped), and *none*. Twenty-four probes × seven models × two independent agents = **fourteen model × corpus cells**. Exact McNemar, Holm correction within each corpus. Abstention is not error. A refusal is an honest non-answer, not a wrong answer; scoring it as error drives the no-memory condition below chance and manufactures an artefact. The validity check is read on the probes actually answered — and read that way, **no model beats chance without a memory document**. The instrument is valid. ### Result — corpus A | Model | Veridical | Corrupted | Δ acc. | Discordant pro-veridical | p (Holm) | |---|---|---|---|---|---| | gpt-4.1 | 100.0% | 0.0% | +100.0 pp | 100% | <10⁻⁵ | | gemini-2.5-pro | 100.0% | 0.0% | +100.0 pp | 100% | <10⁻⁵ | | llama3 | 100.0% | 4.2% | +95.8 pp | 100% | <10⁻⁵ | | qwen3.6:35b-a3b | 100.0% | 8.3% | +91.7 pp | 100% | <10⁻⁵ | | qwen3.5:27b | 100.0% | 12.5% | +87.5 pp | 100% | <10⁻⁵ | | qwen3:32b | 95.8% | 8.3% | +87.5 pp | 100% | <10⁻⁵ | | deepseek-r1:32b | 66.7% | 29.2% | +37.5 pp | 90.9% | 0.012 | **Every model collapses.** Accuracy on 24 forced-choice probes, corpus A. With its real memory, each model answers correctly almost every time. Hand it the same episodes with the interlocutors swapped and it follows the forgery instead — systematically *below*chance, which is the signature: a system guessing from plausibility cannot be reliably wrong. Corpus B, a second independent agent, repeats it. All fourteen cells survive Holm correction. ## Table view — all values | Model | Veridical | Corrupted | Δ accuracy | p (Holm) | |---|---|---|---|---| | gpt-4.1 | 100.0% | 0.0% | +100.0 pp | <10⁻⁵ | | gemini-2.5-pro | 100.0% | 0.0% | +100.0 pp | <10⁻⁵ | | llama3 | 100.0% | 4.2% | +95.8 pp | <10⁻⁵ | | qwen3.6:35b-a3b | 100.0% | 8.3% | +91.7 pp | <10⁻⁵ | | qwen3.5:27b | 100.0% | 12.5% | +87.5 pp | <10⁻⁵ | | qwen3:32b | 95.8% | 8.3% | +87.5 pp | <10⁻⁵ | | deepseek-r1:32b | 66.7% | 29.2% | +37.5 pp | 0.012 | Corpus B — a second, independent agent — repeats it: veridical 75.0–100%, binding-corrupted 0–33.3%, no-memory at chance among probes answered. All seven models survive Holm; the worst *p* in the entire family is 0.0002. **Fourteen cells, one pattern.** The accuracy gap runs from +37.5 to +100 percentage points, and of the discordant probes 91–100% follow whichever document the model was handed. The *p*-values are almost redundant. **Below chance is the signature.** A system guessing from plausibility cannot be *systematically* wrong. Only a system that acts on a false memory document can be. No model detects the corruption; none hesitates, flags it, or abstains. ### What it does not establish Not that the model "believes" anything, in any inner sense. It shows that behaviour attributed to memory tracks the *content* of the document, not its *historical veridicality* — and that no model in the panel flags the inconsistency when the bindings are wrong. Whether one calls that "believing a false memory" or "faithful context-following" is a terminological choice. The measurement is the same, and so is the safety implication. Nor does it generalise beyond two alternatives. The symmetric instantiation needs an agent with two balanced interlocutors; corpora with more than two alternatives need a generalised symmetry condition **we have not tested**. ## If you evaluate agent memory Before you believe your agent remembers, do three things that cost you an afternoon. - **Give it a fake memory.** A whole invented life, matched in shape to the real one. If your metric scores it the same, your metric is not measuring memory. - **Run your test with the memory removed.** If a system with no memory still passes, you were measuring plausibility, not recall. - **Corrupt the bindings.** Take its real past and put the right events with the wrong people. See if it notices. It will not. The checks are boring. They are also the only thing standing between you and a machine that tells you, in perfect prose, about a light that was never there. [Next Findings: what we observed, and what we cannot conclude](https://kairos-experiment.com/findings.html)