Hallucination Is an Architecture Problem, Not Only a Prompt Problem

If you've built a knowledge base on top of an LLM, you've probably noticed something counterintuitive: the outputs rarely contain flat-out fabrications. What they contain is drift — the model summarizes accurately but slightly overstates a confidence level, rounds a number, or drops a qualifier. "As of 2023" quietly becomes "currently." A causal relationship is inferred from correlation.

These are small errors individually. Across a knowledge base that gets queried repeatedly, they compound.

Prompt instructions help. Don't make things up, answer only from the provided context, never fabricate — these do nudge model behavior in the right direction. We use them too. But they leave enforcement entirely up to the model doing the generating. A language model producing text token-by-token has no reliable internal mechanism to audit its own faithfulness to a source.

We built Synthadoc — an LLM-based knowledge base compiler — around a different bet: hallucination is an architecture problem, not only a prompt problem. A prompt fix is hoping the model behaves. An architecture fix is verifying that it did.

The approach has three distinct layers, each targeting a different failure mode:

None of these is a complete solution alone. Together they make hallucination detectable and traceable rather than just discouraged.

Every wiki in Synthadoc has a purpose.md

that defines its domain. When a new source arrives, the ingest agent reads this before deciding what to do with it:

Wiki scope (from purpose.md):
{content of purpose.md}

action="skip" means the source is completely OUTSIDE the wiki's domain.
…

The agent then chooses: skip

, create

, update

, or flag

. Off-domain content is dropped at the door.

When sources conflict, the flag path runs: the ingest agent issues a flag

action and the page moves to a contradicted

lifecycle state, automatically excluded from retrieval until the conflict is resolved. A contradicted page is one where the system can't be confident what's true — serving answers from it would be hallucination with extra steps.

Resolving a contradiction requires an explicit human approval. An interactive agentic workflow surfaces the conflict, shows a diff of the proposed edit, and blocks writing until a human accepts it. The gate is unconditional.

At query time, Synthadoc retrieves the most relevant pages using hybrid search (BM25 + optional vector re-ranking) and builds a synthesis prompt that includes those pages verbatim:

Answer using ONLY these wiki pages. Cite with [[PageTitle]].
Extract and include all specific facts — dates, years, numbers, and names —
even when they appear briefly or in passing.

Question: {question}
Pages: {retrieved_wiki_pages}

The purpose.md

domain description is prepended to the pages block as a reminder during synthesis.

When retrieval confidence falls below threshold, the system surfaces the gap rather than guessing. The response notes that the wiki lacks dedicated coverage on that topic and generates enrichment suggestions — search queries and domain-scoped URLs. Any pages retrieved flow through the normal ingest pipeline and stay in draft

state, out of the search corpus, until a human promotes them to active

.

What grounding doesn't solve: A model can still hallucinate within retrieved content — misattributing which page said what, inventing a specific number close to but not the same as the source, or synthesizing a conclusion that no single page actually states. Grounding reduces the search space. It doesn't eliminate drift.

This is the layer that makes the system verifiable. Each claim in an answer carries a citation marker linking it to specific source lines. A separate LLM pass then audits whether the claim is actually supported:

The auditor sees only the claim and the raw source lines — not the original question. This matters: the same model that wrote the answer is likely to defend it when asked "is this correct?" Separation of duties removes that incentive.

What we actually see: drift

is the most common finding. Specific dates that don't appear in the source, an attribution switched between two people in the same paragraph, a qualifier silently dropped — these are the findings that surface. True hallucination

, where source lines don't support the claim at all, is less common in well-grounded RAG. But it happens, and when it does the audit tells you exactly which source line to check.

Audit results are cached per-page with staleness detection. If a page is re-ingested from updated source material, the cache invalidates and the next audit reflects the new content.

Synthesis across pages. A conclusion drawn by combining facts from multiple pages may be accurate at each step and still misleading in combination. The citation audit operates per-citation — cross-page inferences that no single page actually states fall outside its scope. The practical mitigation: cited pages are always explicit, so synthesized conclusions can be verified. And if a conclusion matters enough to keep, saving it as a wiki page subjects it to future source ingestion — any new source that disputes it triggers the flag

action.

Uniform source error. When all available sources agree on the same wrong answer, no contradiction is detected and Synthadoc answers confidently from incorrect material. Two mechanisms reduce this risk: adversarial lint, which uses the model's general knowledge to flag claims that clearly contradict well-established facts; and re-ingesting from updated authoritative sources, which triggers the flag

action against outdated pages. Neither is a complete solution.

The hallucination debate is usually framed as a model capability question: will the next model be more faithful? For a production knowledge system, that's the wrong frame. Even a perfectly faithful model can only be as accurate as the content it's grounded in. And even a slightly drifting model can be made accountable if its output is systematically audited.

The architecture is a feedback loop: domain scoping controls what enters; lifecycle states control what gets queried; retrieval grounds generation; citation audit classifies the output; the contradiction resolver closes the loop. When a conflict is found, a human resolves it, the page is corrected, and the knowledge base improves. Errors become visible rather than accumulating silently.

That's a more defensible position than any number of ALL CAPS instructions.

Synthadoc is open-source. The citation faithfulness audit, contradiction resolver, and lifecycle system described here are all in the follwoing repository.