How to Flag Stale RAG Documents Before Generation

A developer has published a Python freshness gate that checks retrieved RAG documents against an authoritative revision register and explicit time rules before they reach the generator. The standard-library-only script, check_freshness.py, applies three independent checks — approved revision, effective time, and review deadline — and rejects superseded, not-yet-effective, or overdue records, with the evaluation time passed in as a fixed as_of value for reproducibility. The author notes the gate cannot prove an accepted document is factually correct, only that it is currently approved.

An AI assistant cites a real document and reproduces its instructions accurately. The problem is that the document was replaced last week. The citation proves that a source exists; it does not establish that the source is still approved for the current question. Check retrieved documents against an authoritative revision register and explicit time rules before sending them to the generator. A freshness gate can reject a superseded revision, a document that is not yet effective, or one overdue for review. It cannot prove that an accepted document is factually correct. This tutorial builds that narrow gate in Python. The policy is deliberately specific: the assistant is answering questions about the currently approved material. Historical questions need a different selection policy. Retrieval-augmented generation, or RAG, supplies retrieved material to a language model. A retriever may rank a passage highly because it matches the question even when its lifecycle metadata makes it unsuitable for a current answer. Use three independent checks: | Check | Meaning | Failure response | |---|---|---| | Approved revision | The document matches the current source register | Exclude superseded or unknown records. | | Effective time | The document already applies at the evaluation time | Exclude a future version. | | Review deadline | The record has not passed the team's review boundary | Hold it for review under this policy. | “Review overdue” means the chosen process requires attention. It does not mean a fact becomes false at midnight. Likewise, the newest timestamp does not establish authority. The source owner must decide which revision is approved. Pass the evaluation time into the check. Reading the wall clock deep inside the function makes a saved fixture behave differently as time passes. A fixed as of value gives the test a reproducible meaning. Use timestamps with offsets and normalize them to UTC for comparison. Python's datetime documentation distinguishes aware timestamps from naive ones. This example rejects timestamps without timezone information instead of guessing the author's intent. The revision register must come from the approved document process. Do not let the model infer the current revision from a filename or a passage saying “this is the latest version.” Keep authority separate from the text being evaluated. Use Python 3.12. The example requires only the standard library. Save this as check freshness.py : python from datetime import datetime, timezone def utc value : if not isinstance value, str : raise ValueError "timestamp must be a string" parsed = datetime.fromisoformat value.replace "Z", "+00:00" if parsed.tzinfo is None or parsed.utcoffset is None: raise ValueError "timestamp needs a timezone" return parsed.astimezone timezone.utc def check document doc, current revisions, as of : if as of.tzinfo is None or as of.utcoffset is None: raise ValueError "as of needs a timezone" try: doc id, revision = doc "id" , doc "revision" if any not isinstance v, str or not v.strip for v in doc id, revision : raise ValueError "invalid document identity" effective = utc doc "effective at" review by = utc doc "review by" if review by <= effective: raise ValueError "review by must follow effective at" except KeyError, TypeError, ValueError : return {"allowed": False, "reasons": "invalid metadata" } reasons = approved revision = current revisions.get doc id if approved revision is None: reasons.append "unknown document" elif revision = approved revision: reasons.append "superseded revision" if as of < effective: reasons.append "not effective yet" if as of = review by: reasons.append "review overdue" return {"allowed": not reasons, "reasons": reasons} if name == " main ": now = utc "2026-09-25T08:00:00Z" current = { "note-a": "3", "note-b": "2", "note-c": "5", "note-d": "1", } docs = { "id": "note-a", "revision": "3", "effective at": "2026-09-01T00:00:00Z", "review by": "2026-10-01T00:00:00Z", }, { "id": "note-b", "revision": "1", "effective at": "2026-09-01T00:00:00Z", "review by": "2026-10-01T00:00:00Z", }, { "id": "note-c", "revision": "5", "effective at": "2026-09-01T00:00:00Z", "review by": "2026-09-20T00:00:00Z", }, { "id": "note-d", "revision": "1", "effective at": "2026-10-01T00:00:00Z", "review by": "2026-12-01T00:00:00Z", }, for doc in docs: result = check document doc, current, now status = "ALLOW" if result "allowed" else ", ".join result "reasons" print f"{doc 'id' }: {status}" The code returns reasons instead of one unexplained Boolean. Multiple reasons can be true: a revision might be both superseded and overdue. Invalid metadata fails the gate rather than falling through to approval. The interval is explicit. A document becomes eligible at its effective timestamp. At the exact review deadline, it becomes overdue. Changing either boundary would be a policy change that should be reflected in tests. For an AI document workflow considered by Ranknod, the same gate could distinguish source maintenance work from prompt tuning when an answer repeats an outdated instruction. Run: python3 check freshness.py The observed output on Python 3.12.14 was: note-a: ALLOW note-b: superseded revision note-c: review overdue note-d: not effective yet These are synthetic metadata records. The run verifies the local classification logic; it does not test an actual search index or a model. Additional local checks covered exact deadline boundaries, equivalent timezone offsets, missing authority, invalid date ordering, and missing timezone information. In a real retrieval flow, retain the reason for each excluded result in an appropriately protected diagnostic record. If every useful passage is excluded, return a controlled “current evidence unavailable” outcome or route the task for review. Do not quietly restore the rejected sources just to obtain a fluent answer. It can, when the index and query engine support the required fields. For example, Azure AI Search documents filters that restrict matching records using filterable fields. That capability can help apply an application's selection policy, but it does not create or maintain the policy's source authority. A check after retrieval is still useful at the application boundary, especially when results can arrive through several adapters. It also makes rejection reasons easier to test. Consider both retrieval quality and correctness: filtering a tiny returned list may leave no evidence even when eligible material exists deeper in the index. If your system refetches or reranks results, preserve the same permission and freshness rules. Repeated retrieval should not widen access or relax the current-document requirement without an explicit decision. The gate is only as current as its inputs. If the source register is stale, a replaced document can still pass. If revision identifiers change without the stored text changing correctly, the label may describe the wrong content. Use an ingestion process that ties the text, revision, and metadata together. There is also a timing boundary. A revision could change after the check and before a consequential action uses the answer. Work from a recorded snapshot where appropriate and define when revalidation is required. This small function is not a transaction across the document system and the eventual action. Keep access checks separate and mandatory. A current document is not automatically one the user may read. Keep factual review separate too: approval and freshness do not establish that every sentence is correct. The next debugging question becomes concrete: did the assistant receive an eligible source at the recorded time? Once that is known, the team can investigate the right stage instead of rewriting instructions around an obsolete document.