{"slug": "answer-level-trust-selection-for-physical-vision-language-reasoning", "title": "Answer-Level Trust Selection for Physical Vision-Language Reasoning", "summary": "Researchers propose Answer-Level Trust Selection (ATS), a post-hoc framework that accepts or rejects individual vision-language model (VLM) predictions for physical quantity estimation without fine-tuning or internal logits, using eight behavioral diagnostic scores. Evaluated on Qwen2.5-VL-7B and 20 VLM backbones, ATS identifies stable-but-wrong and prior-tracking failures that self-consistency misses, though improved rejection can reduce retention of correct predictions. The work complements model-level capability evaluation with answer-level reliability assessment.", "body_md": "arXiv:2608.19807v1 Announce Type: new\nAbstract: Vision-language models (VLMs) can estimate physical quantities such as duration, speed, and acceleration from visual observations, but existing benchmarks primarily assess overall model performance against annotated ground truth. In deployment, a key question is whether an individual prediction can be trusted when its ground truth is unavailable. Self-consistency alone may fail to capture important failure modes: a VLM may produce stable-but-wrong estimates or rely on textual priors rather than visual evidence. We formulate answer-level selective prediction for quantitative physical reasoning and propose Answer-Level Trust Selection (ATS), a post-hoc, model-agnostic framework for accepting or rejecting individual VLM predictions. ATS requires no fine-tuning, auxiliary verifier, or access to the model's internal logits. Instead, it aggregates eight interpretable behavioral diagnostic scores derived from repeated queries and controlled interventions into a unified trust score. We evaluate ATS in depth on Qwen2.5-VL-7B and across 20 VLM backbones, examining selective performance, diagnostic behavior, and targeted failure modes. Our results show that intervention-based diagnostics help identify stable-but-wrong and prior-tracking predictions that repeated agreement alone may miss. However, improved failure-case rejection can come at the cost of lower retention of correct predictions. ATS therefore complements model-level capability evaluation with answer-level reliability assessment for quantitative VLM predictions. Code will be released upon publication.", "url": "https://wpnews.pro/news/answer-level-trust-selection-for-physical-vision-language-reasoning", "canonical_source": "https://www.machinebrief.com/news/answer-level-trust-selection-for-physical-vision-language-re-cmsb", "published_at": "2026-08-21 04:00:00+00:00", "updated_at": "2026-08-21 05:42:37.504781+00:00", "lang": "en", "topics": ["artificial-intelligence", "machine-learning", "computer-vision", "ai-research"], "entities": ["Qwen2.5-VL-7B", "Answer-Level Trust Selection", "arXiv"], "alternates": {"html": "https://wpnews.pro/news/answer-level-trust-selection-for-physical-vision-language-reasoning", "markdown": "https://wpnews.pro/news/answer-level-trust-selection-for-physical-vision-language-reasoning.md", "text": "https://wpnews.pro/news/answer-level-trust-selection-for-physical-vision-language-reasoning.txt", "jsonld": "https://wpnews.pro/news/answer-level-trust-selection-for-physical-vision-language-reasoning.jsonld"}}