# Retrieval-Augmented Self-Recall — Part 3: Teaching RAG to Say \"I Don't Know\ > Source: > Published: 2026-07-18 12:06:01+00:00 *Part 3 of Retrieval-Augmented Self-Recall. Code: RE-call. Part 2: hybrid retrieval on Postgres.* Ask your agent *"have we tried this filter on this market before?"* when the honest answer is *never*. A ranking retriever hands back the three closest memos anyway — something about a different filter, on a different market — and the agent, looking at three confident results, concludes: yes, we've looked at this. It just made a decision on a hallucination. Nothing in the stack noticed. No error was raised, because from the retriever's point of view nothing went wrong: you asked for the nearest neighbours and it gave you the nearest neighbours. Everything in Part 2 made retrieval *good*. Good ranking makes this failure **worse**, not better — it returns confident noise faster. This post is about making retrieval *honest*, which for agent memory is the part that actually decides whether you can trust it. So RE-call wraps retrieval in **honesty guards** — this post covers the original three, each answering a question ranking metrics never ask. (The current repo has grown that table to six, and the growth story is [its own post](https://dev.to/gde03/retrieval-augmented-self-recall-what-the-comments-taught-me-re-call-v03-42c1): two of the new guards exist because readers of Part 1 pointed at exactly the weaknesses you're about to see me describe. I'll flag those spots as we go.) `gap_warning` — "is the best match good enough to trust?" After retrieval, look at the best dense cosine similarity. If it falls **below a calibrated threshold**, the top result isn't the answer — it's the least-bad noise. The system sets `gap_warning = true` . The important design idea: this is a **second-order signal**. Retrieval still returns its ranked list; the guard *annotates* how much to trust it. That annotation is what lets the calling agent do something other than blindly act — it can abstain, ask a human to confirm, widen the search, or explicitly note "no prior memory on this" before proceeding. That single flag is the difference between an agent that says *"we've looked at this before"* and one that says *"I have nothing relevant on this — treat it as new."* In a system that makes decisions, that distinction is worth more than any ranking improvement. There's one buried landmine here: **what threshold?** The obvious move is to pick something like 0.50 and move on. That obvious move is quietly, dangerously wrong — and it's the biggest finding in this series, so I'm giving it its own post (Part 5). For now, the load-bearing word is *calibrated*: the threshold is fit to data, never hard-coded. Every memo has a timestamp. The freshness guard reports the **age** of retrieved content and warns when it's stale relative to the re-index cadence (my corpus re-indexes daily, so "stale" has a concrete meaning). This one is specific to *memory* in a way document QA rarely deals with. A documentation corpus is mostly static — last year's page is still roughly true. Agent memory is a moving target: a decision recorded in April may have been *reversed* in June. Without a freshness signal, April-truth and June-truth are indistinguishable at retrieval time, and the agent will happily act on a superseded conclusion. Freshness lets it weight recency, or at least flag the risk. **Honest update, because this section aged:** freshness turned out to be the weakest guard of the three, and a commenter on Part 1 put a finger on why — supersession is a *relation between two memos*, and no per-document timestamp can see a relation. We later measured it: even a steelmanned "trust the newest relevant hit" heuristic still hands back the stale memory **83–100% of the time**, while an explicitly declared `supersedes:` link holds at **0.00**. The fix (a trust layer that binds the relation at write time) and the measurement are in [the follow-up post](https://dev.to/gde03/retrieval-augmented-self-recall-what-the-comments-taught-me-re-call-v03-42c1). The most agent-specific guard of the three. Before the agent proposes an idea, it queries memory for **closed decisions** on that topic — the "we tried X, it failed, here's why" memos — and the guard surfaces them. The failure it prevents is subtle and expensive: an agent re-proposing a dead idea because the memo that killed it three months ago didn't happen to rank in the top results for today's phrasing. Ranking-optimized retrieval is bad at this specifically, because a settled-decision memo is often *lexically* distant from the fresh proposal even though it's the most decision-relevant document in the store. The implementation leans on structure: decision-type memos (closed hypotheses, postmortems) are typed, and a targeted retrieval path prioritizes them when the agent is in "propose" mode. Memory that can't defend its own past decisions is condemned to relive them. Retrieval answers one question: *what's closest?* The guards answer the three that actually govern whether the agent should act: `gap_warning` )That's the whole difference between a **search index** and a **memory**. A search index ranks. A memory knows its own limits. (Since this was drafted, the guard table grew: trust verdicts with declared supersession, an opt-in entailment judge for the high-similarity-but-wrong case a threshold can never catch, and a write-time lint for the supersession graph. All three exist because readers argued with this post's ancestors — that story, with measurements, is [the follow-up](https://dev.to/gde03/retrieval-augmented-self-recall-what-the-comments-taught-me-re-call-v03-42c1).) A guard only helps if its signal **reaches the decision layer**. A `gap_warning` that gets computed and then dropped before the agent sees it is worse than useless — it's false assurance that the system is careful when it isn't. So in RE-call the honesty signals ride *inside* the retrieval result: you cannot get the answer without also getting "here's how much to trust it." (If you read the applied series, this is the same principle as making the *tool* enforce the rule instead of trusting the prompt to.) The guards make claims: *this is a gap, this is stale.* Claims demand measurement — and "how well does it know when it doesn't know?" is a metric that standard RAG benchmarks don't even have. Part 4 builds the eval harness that measures it, and delivers the first finding about which pipeline components actually earn their cost. *Part 3 of Retrieval-Augmented Self-Recall. Code: RE-call. This is the layer that makes Claude Code, Beyond the Prompt's memory trustworthy, not just searchable.*