# Your AI cited a real file. It still lied to you.

> Source: <https://dev.to/enhanciar_ai_96a4ba4877e3/how-we-make-an-llm-cite-every-answer-down-to-fileline-and-check-the-citation-is-telling-the-truth-hmj>
> Published: 2026-09-22 16:39:32+00:00

**TL;DR** — A citation proves a document was *retrieved*. It does not prove the sentence next to it came from there. We built Enhanciar to (1) force every answer to cite, (2) resolve each citation to a repo file and line, and (3) re-read the cited source and grade each claim `supported / partial / unsupported / unverifiable`. The interesting part is not the LLM — it's the free, deterministic checker that runs *before* the LLM, and the rule that the verifier must never say "supported" just because it couldn't run.

Every "chat with your codebase" tool now prints little `[1] [2]` markers under its answers. Ask one of them *"which service handles refunds for cancelled orders?"* and you'll get a confident paragraph, a source link, and a warm feeling.

Then you click the link and the file doesn't mention refunds.

This is the failure mode we kept hitting when we tried to use these tools on real engineering teams: **the model cites a real page and then paraphrases something the page does not say.** Retrieval worked. The citation is "valid". The answer is still wrong, and the citation makes it *more* convincing, not less.

We wanted a tool an engineer could trust on their first day at a company — where "trust" means "I can click through and see the line that backs this sentence, or the tool tells me it couldn't find one."

Here's how we built it. It's three layers, and the order matters.

We don't index raw files for Q&A. At ingest time we build a **wiki**: one Markdown page per meaningful unit (a service, a route, a module, a decision), each with frontmatter that records *exactly which repo files the page was built from*. Something like:

```
---
slug: entities/cart-service
sources:
  - src/services/cart.ts
  - src/routes/cart.ts
---
# cart-service
Handles cart mutation and checkout hand-off. Exposes `POST /cart/items` ...

## Dependencies
- [[entities/pricing-service]]
```

Answers are generated *over these pages*, and the model is required to cite them with wiki-style handles:

```
Refunds for cancelled orders are handled by the payments worker
[[entities/payments-worker]], which is triggered from the order
state machine [[flows/order-cancellation]].
```

We parse those handles out of the finished answer with a boring regex (we tolerate a `§ heading` suffix and a trailing `.md` because models add them anyway, dedupe, and cap at 16). Every handle resolves to a page, and every page resolves to `file(s)` in the repo. So one hop from `[[entities/payments-worker]]` gets you to `src/workers/payments.ts`, and the AST index we build at ingest gets you to a line.

That's the "file:line" part. It's the *easy* part, and it's where most tools stop.

Before an answer ever cites a page, the page has to survive a **per-claim verification** pass. We break each generated page into individual claims and adjudicate each one, *cheapest oracle first*.

A surprising share of claims are **structural**:

`generateMetadata`"`[[get-meta-details-service]]`"` GET /aboutUs`"
We already extracted all of that with tree-sitter at ingest time. So these claims are settled by *lookup* — no model call, no API key, no tokens. If the AST says `generateMetadata` is defined at `app/layout.tsx:42`, the claim is `supported` with evidence `{file, line}`. If the symbol doesn't exist anywhere, it's `unsupported`.

Anything tier 1 can judge, tier 1 judges. Tier 2 never sees it. That's the whole cost story.

Prose claims — *"this service handles refunds for cancelled orders"* — have no structural handle. A model has to re-read the cited source and say whether it supports the sentence. It must return a **quoted span** and the line it was found at, or it doesn't get to say `supported`.

Every claim ends up with exactly one of:

| Verdict | Meaning | 
|---|---|
| `supported` | evidence found for the *specific* assertion | 
| `partial` | something related found, but not what was asserted | 
| `unsupported` | the cited source was checked and **does not say this** | 
| `unverifiable` | nothing could check it — no citation, source gone, checker unavailable | 

**`unverifiable` is load-bearing and is never a synonym for "fine".** Early on, tier 1 had a bug where it reported every React hook in a repo as `unsupported`. The lesson stuck: if the oracle has no data, or the critic has no key, the claim stays `unverifiable` and the UI says what was *not checked*. Silently marking things `supported` because the checker broke would be the worst possible failure of the feature, so every "cannot judge" path returns `unverifiable` rather than a guess.

There was a real design fork here: drop unverified claims, or tag them? Tagging won. A wrong verifier that silently destroys real information is far worse than a wrong verifier that mislabels it — a tag is inspectable and reversible. So a page can carry a sentence flagged `unsupported` and you can see it, click through, and decide.

Layer 2 settles "is this page true about the code". It does *not* settle the failure I opened with: the model cites `[[cart-service]]` while paraphrasing something that page doesn't say.

So we run the **same** checker a second time, with one substitution: for a wiki page the source is the repo file; for an answer the source is the wiki pages it cites. Same verdict vocabulary, same evidence discipline (quoted span + page + line), same extractor, same free oracle, same paid critic. It is deliberately *not* a second verification system.

Four properties, in the order they matter:

`unverifiable` ≠ `unsupported`.` supported` because the checker broke.
Ask through the MCP server (Claude Code, Cursor, Claude Desktop all work). Rendered here as plain text — the UI shows the same verdicts as badges:

```
> which service handles refunds for cancelled orders?

Refunds for cancelled orders are issued by the payments worker
[[entities/payments-worker]] ✅ src/workers/payments.ts:118
when the order state machine emits `order.cancelled`
[[flows/order-cancellation]] ✅ src/orders/machine.ts:64.
Partial refunds are not supported ⚠️ unverifiable — no cited page discusses partial refunds.
```

Two sentences are backed by a file and a line you can open. The third is honestly labelled: the model asserted it, nothing in the cited pages says it, so you don't get a green tick. That third line is the feature.

Tier 1 is free forever. Tier 2 and answer-verification spend the user's own key (BYOK), once per page at ingest and once per answer if the flag is on — it's **off by default** precisely because it costs something, and the token counts come back with the verdicts so you can see what you paid for.

If you're building anything RAG-shaped for engineers: the citation is the *start* of trust, not the end. Re-read the source. Grade the claim. Say "I couldn't check this" out loud.

We're building this as [Enhanciar](https://enhanciar.in?utm_source=devto&utm_medium=article&utm_campaign=launch) — a company brain for engineering teams that answers with receipts. It's waitlist-only right now; if you want to try it on your repo, the link is on the site. Happy to answer questions about the verifier design in the comments.
