cd /news/ai-safety/hallucination-detector-rankings-inve… · home › topics › ai-safety › article
[ARTICLE · art-170429] src=snipvote.com ↗ pub= topic=ai-safety verified=true sentiment=↓ negative

Hallucination detector rankings invert between Llama-3.2-3B-Instruct and Qwen2.5-3B-Instruct models

A paper posted to arXiv (2611.17339) reports that rankings of hallucination detectors invert between Llama-3.2-3B-Instruct and Qwen2.5-3B-Instruct, with a Spearman rho of -0.80 and a Kendall tau-b of -0.667. The authors conclude that detector performance does not generalize across small, 4-bit quantized on-device models, so practitioners must re-evaluate detectors on their specific deployment target rather than relying on rankings from 7B+ research models. The finding matters because a detector chosen on another model's ranking can silently fail or waste GPU cycles in production.

read1 min views4 publishedNov 16, 2026
Hallucination detector rankings invert between Llama-3.2-3B-Instruct and Qwen2.5-3B-Instruct models
Image: Snipvote (auto-discovered)

arXiv

Hallucination detector rankings invert between Llama-3.2-3B-Instruct and Qwen2.5-3B-Instruct models

Which summary reads better? Pick one — models revealed after.Both summaries are AI-generated.

Cross-model hallucination detector rankings actively invert with Spearman rho of -0.80 and Kendall tau-b of -0.667, rendering detector performance unreliable across small, on-device LLMs. This means practitioners cannot rely on detector rankings from larger models and must rigorously evaluate detectors on their specific deployment targets to avoid fairness-of-error-rates issues in production systems.

Hallucination detector rankings invert completely (Spearman rho -0.80) when moving from 7B+ research models to 2–3B 4-bit on-device models. This means the detector you pick for production will either save or waste GPU cycles and user trust depending on which small model you ship—no existing method reliably generalizes. Re-test every detector on your exact quantized model or risk silent failures in the wild.

── more in #ai-safety 4 stories · sorted by recency
── more on @arxiv 3 stories trending now
sponsored brought to you by zahid.host 4,200+ EU-deployed projects
reading about agents? ship yours in a single git push.

Run your AI side-project on zahid.host

EU-based hosting, git-push deploys, automatic HTTPS, no cold starts. Free tier with a custom domain — perfect for shipping the agent you just read about.

$git push zahid main
→ Live at https://your-agent.zahid.host ✓
Get free account → Pricing
from €0/mo · no card required
LIVE [news/hallucination-detect…] indexed:0 read:1min 2026-11-16 · —