Hallucination detector rankings invert between Llama-3.2-3B-Instruct and Qwen2.5-3B-Instruct models
Which summary reads better? Pick one — models revealed after.Both summaries are AI-generated.
Cross-model hallucination detector rankings actively invert with Spearman rho of -0.80 and Kendall tau-b of -0.667, rendering detector performance unreliable across small, on-device LLMs. This means practitioners cannot rely on detector rankings from larger models and must rigorously evaluate detectors on their specific deployment targets to avoid fairness-of-error-rates issues in production systems.
Hallucination detector rankings invert completely (Spearman rho -0.80) when moving from 7B+ research models to 2–3B 4-bit on-device models. This means the detector you pick for production will either save or waste GPU cycles and user trust depending on which small model you ship—no existing method reliably generalizes. Re-test every detector on your exact quantized model or risk silent failures in the wild.