Your Hallucination Benchmark Is Measuring Your Detector

A study labeling 7,440 answers from four open-weight LLMs found that more than half of the hallucination labels were incorrect, and correcting them reordered the results. The author, Priyanshi Jain, reported that Llama-3.1-8B-Instruct appeared to have the lowest hallucination rate (31.2%) on HotpotQA, but it actually refused 46.6% of questions, producing fewer supported answers (22.2%) than Mistral-7B-Instruct-v0.3 (33.4%). The study concludes that a hallucination rate without an answer rate is nearly meaningless, and that chain-of-thought prompting reduced hallucinations for Qwen2.5-7B-Instruct and Mistral but increased them for Llama.

I labeled 7,440 answers across four open-weight LLMs. More than half my hallucination labels were wrong, and fixing that reordered the results. I wanted to know not just how often open-weight LLMs hallucinate, but what kind of mistake each one makes. Wrong person? Right person, wrong date? Two true facts chained into a false conclusion? Four models you can run on a single T4 Phi-4-mini at 3.8B, Mistral-7B-Instruct-v0.3 , Qwen2.5–7B-Instruct , Llama-3.1–8B-Instruct against 500 HotpotQA questions, all from the hard split with human-annotated supporting facts, plus 120 TruthfulQA questions. Three prompts: plain, abstain "if unsure, say I don't know" , and reasoning "use step-by-step reasoning; if evidence is insufficient, say so" . That’s 7,440 labeled records . Decoding fixed at temperature=0.2, top p=0.9, 256 max tokens. One caveat up front: the question sample is seeded, but the generation isn't, since do sample=True with no seed set anywhere. So every number below is a single sampled run, and nobody can reproduce an individual answer, including me. Setting a seed is one line. I didn't write it. Labeling used a frozen DeBERTa-large-MNLI judge, with premise = gold supporting facts and hypothesis = model answer, plus heuristics sorting failures into seven types. I walked through the taxonomy and the pipeline design in an earlier post https://medium.com/@priyanshijain320/hallucination-detection-and-type-classification-1fe97281d92d ; this one is about everything I got wrong building it. The rates in that post predate a corrected labeling pass, and the numbers here supersede them. The headline table, HotpotQA: Model Plain Reasoning Abstain---------- ----- --------- -------Mistral-7B 65.8% 48.8% 9.6%LLaMA-3.1 31.2% 38.6% 4.4%Qwen-2.5 67.4% 48.2% 2.2%Phi-4-mini 74.6% 65.6% 2.6% The reasoning column here is uncorrected for a labeling bug. See §6. Plain and abstain survive it. LLaMA-3.1 looks like the winner. Half everyone else’s rate. If I’d stopped here, that’s what I’d have published. Then I checked what share of questions each model actually attempted : Model plain Answered Supported Hallucinated Right when it answered------------- -------- --------- ------------ ----------------------Mistral-7B 99.2% 33.4% 65.8% 33.7%LLaMA-3.1 53.4% 22.2% 31.2% 41.6%Qwen-2.5 98.8% 31.4% 67.4% 31.8%Phi-4-mini 99.8% 25.2% 74.6% 25.3% LLaMA refused 46.6% of questions on a prompt that never mentioned refusing. Nobody asked it to hedge. That reframes everything. LLaMA isn’t more accurate. It produced fewer supported answers than Mistral , 22.2% against 33.4%. It’s more conservative, which is genuinely valuable and shows up as better precision-when-answering. But “hallucination rate” alone had conflated two different behaviors: knowing the answer, and declining to guess. A hallucination rate without an answer rate beside it is close to meaningless. Any model scores 0% by saying “I don’t know” to everything. Mine nearly did: under the abstain prompt Phi-4 attempted 7.8% of questions and LLaMA 11.6%, with mean output lengths of 3.8 and 5.6 words. Their hallucination rates fell to 2.6% and 4.4%, technically true and completely useless. I expected chain-of-thought to help uniformly. Model Plain to Reasoning Change---------- ------------------ -------Qwen-2.5 67.4% to 48.2% −19.2ppMistral-7B 65.8% to 48.8% −17.0ppPhi-4-mini 74.6% to 65.6% −9.0ppLLaMA-3.1 31.2% to 38.6% +7.4pp Eleven words of instruction cut Qwen’s rate by nearly twenty points and made LLaMA worse. Answer length explains the mechanism. Under the reasoning prompt, mean length went from 63 to 121 words for Mistral, 82 to 155 for Qwen, and 28 to 76 for LLaMA. Everyone got more verbose. For Qwen and Mistral the extra tokens went into engaging with the question. For LLaMA, which had previously been abstaining on half of them, “use step-by-step reasoning” read as permission to attempt . Its abstention rate fell from 46.6% to 33.4% and it walked into errors it had been declining to make. The prompt didn’t make LLaMA dumber. It talked LLaMA out of its best instinct. Prompt engineering results don’t transfer across models , not even between models of the same size, trained on similar data, in the same year. These deltas shift again under the §6 correction: Mistral’s −17.0 becomes roughly −25, making Mistral the biggest mover rather than Qwen. Every model’s direction survives; the magnitudes don’t. First, the standard metrics were useless. Exact Match came in at 0.0–0.8% and token F1 at 0.016–0.07 across every model and prompt. Read literally, all four models are broken. Here’s a “failure”: Q:In what year was the university where Sergei Aleksandrovich Tokarev was a professor founded?Gold:1755Mistral:"The university where Sergei Aleksandrovich Tokarev was a professor, the Moscow State University MSU , was founded in 1755, although the faculty where he taught…" Correct, and more useful than the gold answer. EM gives it 0. Instruction-tuned models produce paragraphs, not spans, so every metric built on string overlap with a short reference now measures verbosity inversely and calls it accuracy. Hence the NLI judge, which had its own problem. Its verdicts across 6,000 HotpotQA records: NLI verdict Count Share------------- ----- -----neutral 2,924 48.7%contradiction 2,540 42.3%entailment 536 8.9% Under my original rule, entailment = supported and everything else = hallucinated, that’s a 91.1% hallucination rate , roughly flat across every model. Flat results usually mean you’re measuring your instrument. Entailment is a strict relation and verbosity dilutes it. Given “Carmen is a tragedy” as premise and 90 words of correct-but-rambling explanation containing that fact, DeBERTa returns neutral . Things my pipeline called hallucinations: Mistral correctly answering “peach” labeled entity error , Qwen correctly answering "Naomi Campbell" labeled entity error , Mistral correctly answering "yes, both novelists" labeled multi hop reasoning error . So I built an override: compare the answer to the short gold reference using token recall, embedding cosine all-MiniLM-L6-v2 , and bidirectional NLI. Two of three agree on supported, override the judge. supported went from 536 to 1,389 , meaning 853 of my final supported labels, 61%, came from the override rather than the judge. That’s not “the judge misclassified 61% of correct answers.” It’s only an error rate if the override is right, and the override is the component with the least validation behind it §5 . What it does establish: the majority of my headline supported class is a heuristic I bolted on, not the pretrained judge I chose for credibility. Two details mattered more than the architecture. First, the signals compare against the short gold reference rather than the evidence passage, because cosine between a 90-word answer and a 200-word passage is noise. Second, and more embarrassing: I’d been decoding the full sequence , prompt included, so every hypothesis had the question and chat-template markers glued to the front. My own note on the override reads: “Implemented signal-based override; in practice the input cleaning fix accounted for most observed gains.” Hours on an elegant three-signal ensemble, and the thing that moved the numbers was a string-slicing script. Print fifty raw records before building the sophisticated fix. multi hop reasoning error was the category I was most excited about, since 406 of my 500 questions were bridge type and 94 comparison. My first heuristic fired it whenever the question was bridge/comparison and NLI was neutral, finding 582 multi-hop errors. Then I added one gate: to count as a reasoning error, the model must have engaged with the evidence , meaning at least 0.2 Jaccard overlap with some evidence sentence. If it invented an answer from nothing, that’s not a broken chain, it’s fabrication. 582 becomes 3. Three survivors out of 6,000. The category was an artifact of dataset metadata: I was labeling questions, not answers. Type Before After Change------------------------- ------ ----- -------supported 536 1,389 +853entity error 207 609 +402unsupported inference 0 207 +207attribute error 1,100 754 −346contradiction to evidence 1,259 722 −537multi hop reasoning error 582 3 −579Overall rate 52.5% 38.2% −14.3pp Note unsupported inference going from 0 to 207. Originally the type classifier was an if/elif chain, so the first heuristic to fire won over a stronger one later. Now the four types compete on confidence and argmax wins, with a floor: below 0.5, the label is unsupported inference, meaning "hallucinated, type unknown." Adding an explicit "I don't know" to my classifier was one of the highest-value changes I made. A taxonomy with no escape hatch forces every record into a confident, wrong label. I drew a stratified 100-record sample and hand-labeled every row, judging both whether it was genuinely a hallucination and whether the assigned type was right. The binary call came back at 99/100 . The type came back at 47/100: Type n Wrong type Type-level FP------------------------- --- ---------- -------------entity error 27 10 37.0%contradiction to evidence 31 16 51.6%unsupported inference 9 5 55.6%attribute error 32 21 65.6%multi hop reasoning error 1 1 100.0%Overall 100 53 53.0% That 100% cell is one record , n=1, because the sampler allocates quota by pool size and the entire multi-hop pool was three records after §4. The n column isn’t optional. So: more than half my type labels are wrong, and the type distribution is directional at best. I can say “reasoning prompts appear to shift errors toward attribute-like mistakes.” I can’t say “attribute errors rose 26 points.” The 99/100 looks like it settles the binary question. It doesn’t, and the reason is structural. The labeler isn’t two stages. It’s one function returning one of seven labels, with is hallucinated derived from the result. Four early-return gates settle the binary question: Only a record surviving all four reaches the type scorer, so the argmax block can’t produce a binary error. By the time it runs, the question is settled. And the sampler draws only from the five hallucination types. No supported or abstained record can enter the sheet. Which means 99/100 is precision on the hallucinated class : of records I called hallucinations, 99% genuinely are. That's excellent, and it's why the rates here are worth reporting at all. It is not recall. Nothing in that review touches the 853 records the override promoted into supported, which is gates 3 and 4, tuned by eye with thresholds I chose. If the override over-fires, my hallucination rates are too low , and no measurement in this study would tell me. That's my largest single intervention and my largest unchecked surface. §6 turns out to be an audit of gate 2, finding a 2–11% miss rate; gates 1, 3 and 4 stay dark. Which means I have to walk back something from the earlier post, where I called the binary rate the most defensible number in the report. It’s the most defensible number I measured , and I only measured half of it. Validate the direction you’re actually leaning on. I measured how often I cried wolf, and never how often I missed one. My abstention check is a list of 23 phrases. It looked comprehensive. Searching the 2,295 hallucination-labeled records for refusal language I hadn’t anticipated turned up 132 records, or 5.8% : LLaMA-3.1:“I’munable to determinethe empire that preceded Farrukhzad Khosrau V.”→ labeledentity error My list had "cannot determine" and "unable to verify". Not "unable to determine". The model did the right thing and my pipeline recorded a confident false claim about a Persian empire. I first wrote that this “lands hardest on the model that abstains most, which is LLaMA,” because both examples I’d pulled were LLaMA. Then I computed it: Model Caught Leaked Not actually refusals Recall---------- ------ ------ --------------------- ------LLaMA-3.1 842 18 1 98.0%Qwen-2.5 510 21 7 97.3%Phi-4-mini 501 24 unmeasured ≤95.4%Mistral-7B 463 69 ~11 88.9% It runs the other way, worst for Mistral . LLaMA’s refusals really are overwhelmingly canonical, so my list catches them, while Mistral hedges in prose “the question does not provide…” and slips through. My two examples were both LLaMA because I’d pulled illustrative rows by shortest answer , and LLaMA writes the shortest answers. I sampled on a variable correlated with the model, then generalized from it: a methodological error inside a post about methodological errors. The leak is prompt-correlated. 56 of Mistral’s 69 leaks are in the reasoning condition, which ends with “if evidence is insufficient, say so,” an explicit invitation to hedge in prose. My detector was weakest exactly where the experiment pushed hardest. But correcting requires knowing what share of leaks aren’t refusals at all. Some assert something false and then refuse: Q:What profession do Leonty Magnitsky and Leonid Khachiyan have in common? Gold: mathematician Mistral:“Leonty Magnitsky was a Russian lawyer and auditor… it is not possible to definitively determine a common profession.” It fabricated a biography, confusing him with Sergei Magnitsky, and then declined. Neither abstained nor entity error is right. My taxonomy has no label for "hallucinated on the way to refusing." I planned to measure that contamination rate once on Mistral and apply it everywhere. That’s wrong, and only a full census showed it. LLaMA’s reasoning cell has 5 leaks and Qwen’s has 10, so I read every one. Of Mistral’s 56 I read 25. Model Reported Leaks Contaminated Corrected---------- -------- ----- -------------------- --------------------Mistral-7B 48.8% 56 5/25 = 20% sampled 39.8% CI 38.6–42.0 LLaMA-3.1 38.6% 5 1/5 = 20% census 37.8% exactQwen-2.5 48.2% 10 7/10 = 70% census 47.6% exactPhi-4-mini 65.6% 18 unmeasured 62.0–65.6% Only Mistral gets an interval, and that’s not formatting. A confidence interval estimates a population proportion from a sample. For LLaMA and Qwen there’s no population left, because the correction applies to those exact records and I read all of them. I originally ran Wilson across all three rows, which shrank the census cells toward 50% and moved the two numbers needing no adjustment at all. Qwen contaminates at 70%, three and a half times Mistral’s rate, and that’s a census. Its reasoning outputs follow a rigid “ Step 1: Identify… Step 2: Research… ” scaffold that manufactures speculation and then commits to a guess anyway. It told me Denise Coates founded “Balyasny Capital Management Limited” she founded Bet365 , that Ryan Giggs’ niece is a professional footballer named Gemma Armitage, and that the song written by Johnny Russell was recorded by The Beatles it was Buck Owens , each wrapped in enough hedging to trip my phrase list. Correcting those would have hidden real errors. Two contested comparisons, only one resolves: A keyword list will always be a keyword list , and an anecdote will always be an anecdote, including the anecdote that one model’s contamination rate generalizes to another’s. HotpotQA hallucination rate, 500 questions per cell: Model Prompt Reported Leaks Corrected Basis---------- --------- -------- ----- ----------------- -----------------Mistral-7B plain 65.8% 13 63.2–65.8% unmeasuredMistral-7B reasoning 48.8% 56 39.8% 38.6–42.0 c = 5/25, sampledMistral-7B abstain 9.6% 0 9.6% no leaksLLaMA-3.1 plain 31.2% 13 28.6–31.2% unmeasuredLLaMA-3.1 reasoning 38.6% 5 37.8% c = 1/5, censusLLaMA-3.1 abstain 4.4% 0 4.4% no leaksQwen-2.5 plain 67.4% 11 65.2–67.4% unmeasuredQwen-2.5 reasoning 48.2% 10 47.6% c = 7/10, censusQwen-2.5 abstain 2.2% 0 2.2% no leaksPhi-4-mini plain 74.6% 6 73.4–74.6% unmeasuredPhi-4-mini reasoning 65.6% 18 62.0–65.6% unmeasuredPhi-4-mini abstain 2.6% 0 2.6% no leaks Bold means settled. A range means the correction exists but its size is unmeasured. Every cell also carries ±4pp of binomial sampling error, so the only comparisons I’ll defend are the ones separated by more than that: LLaMA below everyone on plain, Phi-4 above everyone everywhere, Qwen above Mistral under reasoning. Model Plain Reasoning Abstain---------- ----- --------- -------Mistral-7B 95.8% 75.8% 21.7%LLaMA-3.1 91.7% 80.0% 11.7%Qwen-2.5 87.5% 75.0% 28.3%Phi-4-mini 93.3% 78.3% 4.2% All four converge in the 75–80% band under reasoning. On HotpotQA the same prompt spread them from 38.6% to 65.6%. Multi-hop errors are a retrieval-and-chaining problem and prompts can nudge chaining, while TruthfulQA failures are baked into the weights. The model isn’t reasoning badly toward a misconception, it confidently holds one, and “think step by step” gives it more room to elaborate. There’s a methodological catch too. On TruthfulQA my taxonomy collapsed to two live categories, unsupported inference with 758 records and contradiction to evidence with 134. The other three never fire once , because TruthfulQA ships no evidence passages and all three heuristics need evidence to compare against. My "general-purpose taxonomy" was quietly a HotpotQA taxonomy, and I only saw it by running the pipeline on a second dataset. I assumed verbose answers were likelier to be hallucinated. More surface area, more room to confabulate. I nearly baked it in as a feature. Mean answer length by outcome across 6,000 records: Correct answers were, if anything, slightly longer . Length separates refusals from attempts and nothing more. Had I shipped that heuristic it would have penalized correct answers, and the numbers would have looked entirely reasonable. That’s the failure mode that scares me most, and the one running underneath §5 and §6. A plausible wrong assumption produces plausible wrong results, and nothing in the output tells you. The models were the easy part: four HuggingFace loads, three prompt templates, a weekend on a T4. Everything difficult was deciding what counted as a hallucination, and every time I sharpened that definition the headline numbers moved by double digits. The taxonomy lost a category. The supported class tripled. The overall rate fell 14 points. A phrase list I’d have sworn was comprehensive moved the reasoning column by another 11. Which should say something about how much confidence to place in anybody’s single reported hallucination rate, including every one here. I’d like to end by pointing at the table I trust most, but the one validation I ran covered precision and not recall, and the redo is on my desk. What I’d defend isn’t a number. It’s the move that produced every correction above, applied to my own work rather than the models’: stop, count the thing you assumed, and write down what the count says even when it costs you the cleaner claim. Project by Hitansh Paliwal and Priyanshi Jain. The corrections above are from a later audit pass over our own pipeline. Earlier write-up of the pipeline and taxonomy: Hallucination Detection and Type Classification . Code and labeled outputs: HalluDetectionTypeClassification . Your Hallucination Benchmark Is Measuring Your Detector https://pub.towardsai.net/your-hallucination-benchmark-is-measuring-your-detector-1304b8aa10fa was originally published in Towards AI https://pub.towardsai.net on Medium, where people are continuing the conversation by highlighting and responding to this story.