{"slug": "how-to-flag-stale-rag-documents-before-generation", "title": "How to Flag Stale RAG Documents Before Generation", "summary": "A developer has published a Python freshness gate that checks retrieved RAG documents against an authoritative revision register and explicit time rules before they reach the generator. The standard-library-only script, check_freshness.py, applies three independent checks — approved revision, effective time, and review deadline — and rejects superseded, not-yet-effective, or overdue records, with the evaluation time passed in as a fixed as_of value for reproducibility. The author notes the gate cannot prove an accepted document is factually correct, only that it is currently approved.", "body_md": "An AI assistant cites a real document and reproduces its instructions accurately. The problem is that the document was replaced last week. The citation proves that a source exists; it does not establish that the source is still approved for the current question.\n\n**Check retrieved documents against an authoritative revision register and explicit time rules before sending them to the generator.** A freshness gate can reject a superseded revision, a document that is not yet effective, or one overdue for review. It cannot prove that an accepted document is factually correct.\n\nThis tutorial builds that narrow gate in Python. The policy is deliberately specific: the assistant is answering questions about the currently approved material. Historical questions need a different selection policy.\n\nRetrieval-augmented generation, or RAG, supplies retrieved material to a language model. A retriever may rank a passage highly because it matches the question even when its lifecycle metadata makes it unsuitable for a current answer.\n\nUse three independent checks:\n\n| Check | Meaning | Failure response | \n|---|---|---|\n| Approved revision | The document matches the current source register | Exclude superseded or unknown records. | \n| Effective time | The document already applies at the evaluation time | Exclude a future version. | \n| Review deadline | The record has not passed the team's review boundary | Hold it for review under this policy. | \n\n“Review overdue” means the chosen process requires attention. It does not mean a fact becomes false at midnight. Likewise, the newest timestamp does not establish authority. The source owner must decide which revision is approved.\n\nPass the evaluation time into the check. Reading the wall clock deep inside the function makes a saved fixture behave differently as time passes. A fixed as_of value gives the test a reproducible meaning.\n\nUse timestamps with offsets and normalize them to UTC for comparison. Python's datetime documentation distinguishes aware timestamps from naive ones. This example rejects timestamps without timezone information instead of guessing the author's intent.\n\nThe revision register must come from the approved document process. Do not let the model infer the current revision from a filename or a passage saying “this is the latest version.” Keep authority separate from the text being evaluated.\n\nUse Python 3.12. The example requires only the standard library. Save this as `check_freshness.py`:\n\n``` python\nfrom datetime import datetime, timezone\n\ndef utc(value):\n    if not isinstance(value, str):\n        raise ValueError(\"timestamp must be a string\")\n    parsed = datetime.fromisoformat(value.replace(\"Z\", \"+00:00\"))\n    if parsed.tzinfo is None or parsed.utcoffset() is None:\n        raise ValueError(\"timestamp needs a timezone\")\n    return parsed.astimezone(timezone.utc)\n\ndef check_document(doc, current_revisions, as_of):\n    if as_of.tzinfo is None or as_of.utcoffset() is None:\n        raise ValueError(\"as_of needs a timezone\")\n    try:\n        doc_id, revision = doc[\"id\"], doc[\"revision\"]\n        if any(\n            not isinstance(v, str) or not v.strip()\n            for v in (doc_id, revision)\n        ):\n            raise ValueError(\"invalid document identity\")\n\n        effective = utc(doc[\"effective_at\"])\n        review_by = utc(doc[\"review_by\"])\n\n        if review_by <= effective:\n            raise ValueError(\"review_by must follow effective_at\")\n    except (KeyError, TypeError, ValueError):\n        return {\"allowed\": False, \"reasons\": [\"invalid_metadata\"]}\n\n    reasons = []\n    approved_revision = current_revisions.get(doc_id)\n\n    if approved_revision is None:\n        reasons.append(\"unknown_document\")\n    elif revision != approved_revision:\n        reasons.append(\"superseded_revision\")\n\n    if as_of < effective:\n        reasons.append(\"not_effective_yet\")\n\n    if as_of >= review_by:\n        reasons.append(\"review_overdue\")\n\n    return {\"allowed\": not reasons, \"reasons\": reasons}\n\nif __name__ == \"__main__\":\n    now = utc(\"2026-09-25T08:00:00Z\")\n    current = {\n        \"note-a\": \"3\",\n        \"note-b\": \"2\",\n        \"note-c\": \"5\",\n        \"note-d\": \"1\",\n    }\n\n    docs = [\n        {\n            \"id\": \"note-a\",\n            \"revision\": \"3\",\n            \"effective_at\": \"2026-09-01T00:00:00Z\",\n            \"review_by\": \"2026-10-01T00:00:00Z\",\n        },\n        {\n            \"id\": \"note-b\",\n            \"revision\": \"1\",\n            \"effective_at\": \"2026-09-01T00:00:00Z\",\n            \"review_by\": \"2026-10-01T00:00:00Z\",\n        },\n        {\n            \"id\": \"note-c\",\n            \"revision\": \"5\",\n            \"effective_at\": \"2026-09-01T00:00:00Z\",\n            \"review_by\": \"2026-09-20T00:00:00Z\",\n        },\n        {\n            \"id\": \"note-d\",\n            \"revision\": \"1\",\n            \"effective_at\": \"2026-10-01T00:00:00Z\",\n            \"review_by\": \"2026-12-01T00:00:00Z\",\n        },\n    ]\n\n    for doc in docs:\n        result = check_document(doc, current, now)\n        status = (\n            \"ALLOW\"\n            if result[\"allowed\"]\n            else \", \".join(result[\"reasons\"])\n        )\n        print(f\"{doc['id']}: {status}\")\n```\n\nThe code returns reasons instead of one unexplained Boolean. Multiple reasons can be true: a revision might be both superseded and overdue. Invalid metadata fails the gate rather than falling through to approval.\n\nThe interval is explicit. A document becomes eligible at its effective timestamp. At the exact review deadline, it becomes overdue. Changing either boundary would be a policy change that should be reflected in tests.\n\nFor an AI document workflow considered by Ranknod, the same gate could distinguish source maintenance work from prompt tuning when an answer repeats an outdated instruction.\n\nRun:\n\n```\npython3 check_freshness.py\n```\n\nThe observed output on Python 3.12.14 was:\n\n```\nnote-a: ALLOW\nnote-b: superseded_revision\nnote-c: review_overdue\nnote-d: not_effective_yet\n```\n\nThese are synthetic metadata records. The run verifies the local classification logic; it does not test an actual search index or a model. Additional local checks covered exact deadline boundaries, equivalent timezone offsets, missing authority, invalid date ordering, and missing timezone information.\n\nIn a real retrieval flow, retain the reason for each excluded result in an appropriately protected diagnostic record. If every useful passage is excluded, return a controlled “current evidence unavailable” outcome or route the task for review. Do not quietly restore the rejected sources just to obtain a fluent answer.\n\nIt can, when the index and query engine support the required fields. For example, Azure AI Search documents filters that restrict matching records using filterable fields. That capability can help apply an application's selection policy, but it does not create or maintain the policy's source authority.\n\nA check after retrieval is still useful at the application boundary, especially when results can arrive through several adapters. It also makes rejection reasons easier to test. Consider both retrieval quality and correctness: filtering a tiny returned list may leave no evidence even when eligible material exists deeper in the index.\n\nIf your system refetches or reranks results, preserve the same permission and freshness rules. Repeated retrieval should not widen access or relax the current-document requirement without an explicit decision.\n\nThe gate is only as current as its inputs. If the source register is stale, a replaced document can still pass. If revision identifiers change without the stored text changing correctly, the label may describe the wrong content. Use an ingestion process that ties the text, revision, and metadata together.\n\nThere is also a timing boundary. A revision could change after the check and before a consequential action uses the answer. Work from a recorded snapshot where appropriate and define when revalidation is required. This small function is not a transaction across the document system and the eventual action.\n\nKeep access checks separate and mandatory. A current document is not automatically one the user may read. Keep factual review separate too: approval and freshness do not establish that every sentence is correct.\n\nThe next debugging question becomes concrete: did the assistant receive an eligible source at the recorded time? Once that is known, the team can investigate the right stage instead of rewriting instructions around an obsolete document.", "url": "https://wpnews.pro/news/how-to-flag-stale-rag-documents-before-generation", "canonical_source": "https://dev.to/ranknod/how-to-flag-stale-rag-documents-before-generation-2k08", "published_at": "2026-09-26 14:46:25+00:00", "updated_at": "2026-09-26 15:00:05.116500+00:00", "lang": "en", "topics": ["ai-tools", "large-language-models", "generative-ai", "developer-tools"], "entities": ["Python"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/how-to-flag-stale-rag-documents-before-generation", "markdown": "https://wpnews.pro/news/how-to-flag-stale-rag-documents-before-generation.md", "text": "https://wpnews.pro/news/how-to-flag-stale-rag-documents-before-generation.txt", "jsonld": "https://wpnews.pro/news/how-to-flag-stale-rag-documents-before-generation.jsonld"}}