{"slug": "visuals-lie-consistency-speaks-disentangling-spatial-attention-from-reliability", "title": "Visuals Lie, Consistency Speaks: Disentangling Spatial Attention from Reliability in Vision-Language Models", "summary": "Researchers at arXiv challenge the common assumption that visual attention correlates with reliability in vision-language models. Their VLM Reliability Probe study across multiple models finds that spatial attention metrics have near-zero correlation with accuracy, while self-consistency in generation dynamics is the dominant predictor of truth. The work reveals architectural divergences in how models encode reliability, with implications for building more trustworthy multimodal AI systems.", "body_md": "arXiv:2606.17389v1 Announce Type: new\nAbstract: Multimodal Foundation Models are increasingly used as reasoning agents, making reliability, knowing when a model may hallucinate, critical. A common intuition, which we call the Attention-Confidence Assumption, holds that reliability follows from \"structural\" visual perception: tight attention on relevant regions should signal a trustworthy answer, while scattered attention signals confusion. We challenge this through the VLM Reliability Probe (VRP), a systematic cross-family study of reliability signals in contemporary Vision-Language Models (VLMs). We introduce structural-attention metrics, cluster counts (C_k) and spatial entropy (H_s), to quantify the visual encoder's gaze, and track its evolution (Delta H_s) across layers. This reveals a \"Symbolic Detachment\": models often \"Early Lock\" visual features only to diffuse attention later, severing early perception from final generation. Contrary to the grounding hypothesis, we find a \"Cluster Failure\": spatial attention has near-zero correlation (R approx 0.001) with accuracy. Instead, reliability is a phenomenon of generation dynamics and internal-state distributions. Self-Consistency, the agreement rate across sampled reasoning paths, is the dominant predictor of truth (R = 0.429). Scaling causal interventions exposes a sharp architectural divergence: LLaVA locks its prediction in a fragile late-stage bottleneck, whereas PaliGemma and Qwen2-VL distribute reliability globally, staying resilient even when ~50% or more of their most predictive layer is destroyed. For current VLMs, reliability signals are detached from visual grounding maps and are best inferred from generation-time dynamics and hidden-state probes.", "url": "https://wpnews.pro/news/visuals-lie-consistency-speaks-disentangling-spatial-attention-from-reliability", "canonical_source": "https://arxiv.org/abs/2606.17389", "published_at": "2026-06-17 04:00:00+00:00", "updated_at": "2026-06-17 04:26:23.968576+00:00", "lang": "en", "topics": ["computer-vision", "natural-language-processing", "ai-research", "ai-safety", "large-language-models"], "entities": ["arXiv", "LLaVA", "PaliGemma", "Qwen2-VL", "VLM Reliability Probe"], "alternates": {"html": "https://wpnews.pro/news/visuals-lie-consistency-speaks-disentangling-spatial-attention-from-reliability", "markdown": "https://wpnews.pro/news/visuals-lie-consistency-speaks-disentangling-spatial-attention-from-reliability.md", "text": "https://wpnews.pro/news/visuals-lie-consistency-speaks-disentangling-spatial-attention-from-reliability.txt", "jsonld": "https://wpnews.pro/news/visuals-lie-consistency-speaks-disentangling-spatial-attention-from-reliability.jsonld"}}