Show HN: LLM verification pipeline against hallucinations

VeriQuote, an open-source TypeScript library and CLI released under the MIT license, checks every model citation twice — a deterministic matcher that finds the verbatim quote in the source and an LLM judge that reads the quote against the claim — and passes a citation only if both agree. The project cites a study on clinical guideline questions in which a frontier model quoted its sources word for word for 98% of its claims but only 37% of those claims were fully supported by the quotes. VeriQuote has no dependencies, and its code, benchmarks and raw results are published on GitHub.

VeriQuote · open-source library and CLI Language models put citations like 1 behind their claims, and checking one means opening the source and finding the passage. A recent study on clinical guideline questions found that a frontier model quoted its sources word for word for 98% of its claims, yet only 37% of the claims were fully supported by those quotes. VeriQuote makes the model attach a verbatim quote to every citation and checks each quote twice. A deterministic matcher looks for the quote in the source. An LLM judge reads the quote next to the claim and decides whether it supports it. A citation passes only if both let it through. Pick an example. The matcher compares the quote with the source and ignores the claim. The judge compares the quote with the claim; it sees a little of the source as context but does not check that the quote is really there. Each misses what the other catches. The source texts were written for this demo; the trial they describe is fictional. Source Quote attached Claim in the answer The matcher reruns as you type. The judge needs a model: any OpenAI-compatible endpoint that accepts browser requests, such as OpenRouter. Your key stays in this tab and goes only to that endpoint. The examples above were picked by hand. The results below were not: thousands of generated quotes for the matcher, citations labelled by human annotators for the judge, and four open answering models following the citation instructions. All of it can be rerun from the repository. Hover over a mark to see its numbers. VeriQuote is a TypeScript library and CLI with no dependencies, MIT licensed. Code, benchmarks and raw results are on GitHub https://github.com/rickintoplace/veriquote . Icons by Lucide ISC licence .