RAG Hallucination Diagnosis: Evidence Gating Beats Embeddings for Ask-Your-Docs Chatbot Answers

A developer's architecture decision record recommends that docs chatbots abstain from answering when evidence is insufficient, favoring evidence gating over larger context windows for classifying gaming reports. The two-stage approach retrieves candidate policy passages and gates generation on evidence quality, routing uncertain cases to human review. The post defines failure modes such as retrieval, scope, evidence, and generation failures, and suggests building evaluation sets with realistic report shapes.

Short answer: A docs chatbot should abstain whenever it cannot assemble enough directly relevant evidence for a moderation report. For classifying gaming reports before human review, choose evidence gating over a larger context window: retrieval may propose evidence, but a separate policy must decide whether the system may answer. This favors quality over shaving a little latency from the happy path. The distinction matters because a fluent category label can still be unsupported. Embeddings answer a proximity question. They don't prove that the retrieved passage governs this game mode, policy version, region, or report type. Chunking can preserve more local meaning, and a larger context window can carry more text, yet neither mechanism turns weak evidence into a warranted decision. This architecture decision record chooses a two-stage path: retrieve candidate policy passages, then gate generation on evidence quality and scope. A report that passes the gate receives a suggested moderation category plus citations. A report that fails it goes to human review with a machine-readable reason such as no policy match , scope conflict , or ambiguous evidence . Abstention is a successful outcome, not an exception. Three invariants define the boundary. The answer must cite text that supports the selected category. Every cited passage must carry the policy version and scope used during retrieval. Conflicting passages must not be silently averaged into a confident label. Those rules are more useful than a blanket instruction to "use the context," because they can be tested before and after generation. Keep generation outside the authority boundary. The model can summarize evidence and suggest a label; the moderation service owns the final state transition, validates the response schema, and routes uncertain cases to reviewers. This resembles the discipline needed in OTP delivery: a provider accepting a request doesn't establish that the user received the message. Each boundary needs its own observable result. No citation, no classification. The failure modes are broader than bad chunk size. A current policy can retrieve an obsolete appendix with similar wording. A player report can mention harassment while actually describing impersonation. An audio transcript can lose the proper noun that distinguishes a player from a game item. A long context can contain the correct paragraph and a contradictory paragraph at once. In each case, adding tokens may make the prompt look richer while leaving the decision boundary undefined. OWASP's LLM application guidance is a useful threat-modeling starting point because retrieved content and model output both cross trust boundaries. Start by turning "wrong" into outcomes that an evaluation can distinguish. Retrieval failure means the supporting policy never entered the candidate set. Scope failure means the candidate belongs to the wrong policy version, locale, game mode, or enforcement tier. Evidence failure means the candidate is topically similar but does not entail the proposed category. Generation failure means adequate evidence was present but the output contradicted it, omitted a required citation, or broke the schema. These failures need different repairs. Treating all of them as hallucination leads teams to tune the retriever when the missing component is an authorization rule. Build a small evaluation set from realistic moderation-report shapes, but don't let it become a pile of easy keyword matches. Include paraphrases, short angry reports, mixed allegations, references that depend on earlier conversation, and near-neighbor policies that use the same nouns but prescribe different outcomes. Audio reports deserve their own slice: an open-source speech recognizer can produce the transcript, but the transcript should retain provenance and remain an upstream input, not be mistaken for original evidence. Human reviewers should label the expected category, the supporting policy passages, and whether abstention is acceptable. Then measure the stages separately. Retrieval evaluation asks whether the labeled support appears in the candidates. Gate evaluation asks whether supported cases pass and unsupported or conflicting cases stop. Answer evaluation checks the category and verifies that each citation actually backs the claim. End-to-end accuracy alone hides compensation: a generator may guess correctly after retrieval fails, which looks good in a dashboard and teaches the team nothing useful. I'm not sure one universal similarity threshold exists; corpus vocabulary, embedding model, and policy density change the score distribution. A held-out set and an explicit review of false accepts are what resolve that uncertainty. The practical fix is usually metadata filtering plus evidence checks, not more prompt decoration. Index atomic policy units with their heading path, version, effective period, jurisdiction, and report taxonomy. Retrieve with hard scope filters where the request supplies those fields. Rerank candidates against the actual allegation. Finally, require sufficient support and reject conflicts before calling the generator. Chunk boundaries still matter: keep exceptions and the rule they qualify together, and don't merge unrelated sanctions merely to reach a target token count. | Decision factor | Larger context window | Evidence-gated retrieval | |---|---|---| | Main benefit | Carries more candidate text into one generation call | Makes the permission to answer explicit | | Main risk | Irrelevant or conflicting passages remain available to the model | Conservative thresholds can send more work to reviewers | | Latency shape | More input must travel through the generation path | Retrieval and validation add stages, but abstentions can skip generation | | Best fit | Synthesis where broad recall matters and errors are reversible | Moderation triage where an unsupported label can misroute human review | | Debugging signal | Often reveals only that the final answer was wrong | Separates retrieval, scope, evidence, and generation failures | The quality-versus-latency choice isn't free. Evidence gating adds a reranking or validation step and more telemetry. It can also lower automation when thresholds are cautious. For pre-review classification, that is the right bias: a queue item marked uncertain is visible and recoverable, while a confident but unsupported category can send a report to the wrong workflow. Teams with a low-risk internal search tool, loose synthesis requirements, and users who always inspect sources may reasonably prefer the simpler large-context path. Don't use generation retries as the default response to uncertainty. Retrying the same evidence changes wording more readily than it changes warrant. Retry retrieval only when the next attempt changes a declared variable, such as query decomposition or a scope filter; record that change so the evaluation can tell which path helped. The same principle applies to rate limits in messaging systems — an unexamined retry loop creates load without proving delivery. The critical path below is deliberately generic. The retriever and generator are interfaces, while the moderation policy remains ordinary application code. Thresholds are configuration derived from evaluation, not constants copied from an article. python from dataclasses import dataclass from typing import Protocol @dataclass frozen=True class Passage: text: str source url: str policy version: str scope: str relevance: float class Retriever Protocol : def search self, query: str, filters: dict str, str - list Passage : ... class Generator Protocol : def classify self, report: str, evidence: list Passage - dict: ... def classify report report: str, scope: str, policy version: str, minimum relevance: float, retriever: Retriever, generator: Generator, - dict: candidates = retriever.search report, filters={"scope": scope, "policy version": policy version}, eligible = passage for passage in candidates if passage.scope == scope and passage.policy version == policy version and passage.relevance = minimum relevance if not eligible: return {"status": "review", "reason": "no policy match"} versions = {passage.policy version for passage in eligible} scopes = {passage.scope for passage in eligible} if len versions = 1 or len scopes = 1: return {"status": "review", "reason": "scope conflict"} result = generator.classify report, eligible cited urls = set result.get "citations", allowed urls = {passage.source url for passage in eligible} if not cited urls or not cited urls.issubset allowed urls : return {"status": "review", "reason": "unsupported citation"} return { "status": "suggested", "category": result "category" , "citations": sorted cited urls , "policy version": policy version, } URL membership alone does not prove entailment. The code enforces cheaper structural checks in the request path; semantic support still needs a validator or a constrained category-to-policy mapping, tested against the labeled set. Production code should also log candidate identifiers, filter values, configured threshold version, gate reason, and the final reviewer correction without storing more player content than policy allows. Compliance starts at the event schema, not at the audit dashboard. Watch p50 and tail latency per stage, but pair them with quality signals: retrieval support rate, abstention rate by report type, citation validation failures, and reviewer overrides. A falling abstention rate is not automatically good. If reviewer overrides rise at the same time, the gate has become permissive. Slice results by language, input channel, policy version, and report category so a healthy aggregate doesn't hide an audio-transcript or locale-specific gap. The rejected design sends the top chunks directly to a generator, asks it to answer only from context, and increases the context window when answers drift. It is attractive because the path is short, the demo is easy, and broad prompts can summarize scattered material. The catch is that the design has no enforceable point where weak, stale, or contradictory evidence loses permission to become a classification. Prompt wording carries responsibility that belongs in code. It is not suitable when a wrong category changes routing, priority, or enforcement before a reviewer sees the original report. Stick with the simpler design when the output is exploratory, the user inspects citations before acting, source scope is homogeneous, and abstention machinery would cost more than an occasional reversible error. Evidence gating also has a limit: it cannot repair missing policy coverage or a mislabeled evaluation set. In those cases, improving the corpus and reviewer taxonomy comes first. Ship the gate in shadow mode before it can influence routing. Record what it would accept or abstain on, compare those decisions with reviewer labels, then choose thresholds from the quality target and available review capacity. Roll out by report class, keep a fast disable path, and version the index, filters, prompt, and threshold together. That deployment record turns "the chatbot got worse" into a set of components that can be compared. A bigger window remains a capacity tool. Evidence gating is the decision tool. Further reading: