{"slug": "when-does-uncertainty-measure-rag-hallucination-risk", "title": "When Does Uncertainty Measure RAG Hallucination Risk?", "summary": "A new arXiv paper (2610.22615v1) finds that uncertainty measures hallucination risk in retrieval-augmented generation only when retrieved evidence constrains the answer. Testing four instruction-tuned models on HotpotQA and SQuAD, the authors report that after matching faithful and hallucinated answers within word-count bins, length-only classifiers fall near chance while perplexity reaches 0.81–0.91 AUROC on targeted unsupported edits, and that detection can weaken with only the top-ranked passage before recovering after a second passage is added. A combined uncertainty and semantic-drift score improves calibration, and the measurements remain above chance on RAGTruth QA.", "body_md": "arXiv:2610.22615v1 Announce Type: new \nAbstract: Uncertainty can measure hallucination risk in retrieval-augmented generation (RAG), but only when retrieved evidence constrains the answer. When evidence is present, unsupported factual claims become less likely; when retrieval coverage is poor, the model can answer from internal knowledge and assign high probability to unsupported claims. We test this across four instruction-tuned models on HotpotQA and SQuAD. After matching faithful and hallucinated answers within word-count bins, length-only classifiers fall near chance, while perplexity reaches 0.81--0.91 AUROC on targeted unsupported edits. Varying retrieval depth shows why the score changes meaning: detection can weaken with only the top-ranked passage, then recover after adding a second passage. A combined uncertainty and semantic-drift score improves calibration, and measurements remain above chance on RAGTruth QA. Results support a conditional view of uncertainty as a useful measure of RAG hallucination risk.", "url": "https://wpnews.pro/news/when-does-uncertainty-measure-rag-hallucination-risk", "canonical_source": "https://arxiv.org/abs/2610.22615", "published_at": "2026-10-19 04:00:00+00:00", "updated_at": "2026-10-19 04:20:51.780881+00:00", "lang": "en", "topics": ["artificial-intelligence", "large-language-models", "ai-research", "ai-safety", "generative-ai"], "entities": ["HotpotQA", "SQuAD", "RAGTruth", "arXiv"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/when-does-uncertainty-measure-rag-hallucination-risk", "markdown": "https://wpnews.pro/news/when-does-uncertainty-measure-rag-hallucination-risk.md", "text": "https://wpnews.pro/news/when-does-uncertainty-measure-rag-hallucination-risk.txt", "jsonld": "https://wpnews.pro/news/when-does-uncertainty-measure-rag-hallucination-risk.jsonld"}}