{"slug": "medqa-mm-shortcuts-behind-medical-visual-reasoning", "title": "MedQA-MM: Shortcuts Behind Medical Visual Reasoning", "summary": "A new arXiv preprint (arXiv:2609.03261v1) finds that medical multimodal AI benchmarks overstate visual reasoning because correct answers can be derived from non-visual cues, a phenomenon the authors call 'reasoning inflation.' Across six medical multimodal MCQ datasets, full-input accuracy was 62.63%, but text-only and options-only settings achieved 53.96% and 29.71%, respectively. The authors introduce MedQA-MM, a 1,000-item shortcut-mitigated subset where text-only and options-only accuracy drop to 5.21% and 12.33%, underscoring the need for route-level evidence in medical image-reasoning claims.", "body_md": "arXiv:2609.03261v1 Announce Type: new\nAbstract: A benchmark score credits final answers, but not the route by which an item can be answered. In medical multimodal multiple-choice questions (MCQs), this distinction matters because a correct answer can be supported by the intended image finding or by benchmark-preserved cues in the wording of answers, non-visual clinical text, visible image text, artificial annotations, or device/context artifacts. We call the resulting score-level overinterpretation reasoning inflation. Here, a route is an observable input path that can support answer selection, not a claim about the model's hidden cognition. Across six medical multimodal MCQ datasets, we separate candidate cues from behavioral evidence through prompt- and image-side audits, modality ablations, and matched repairs that preserve the medical target and answer key. In a 13-configuration open-model panel, full-input accuracy is 62.63%, while text-only and options-only settings achieve 53.96% and 29.71%, respectively. Removing length-gap, absolute/conspicuous, and spatial/prepositional cues lowers accuracy by 6.58, 3.50, and 4.77 percentage points. We also construct MedQA-MM, a 1,000-item shortcut-mitigated subset, where text-only and options-only accuracy fall to 5.21% and 12.33%. This does not imply that models never use images; it shows that medical image-reasoning claims require route-level evidence.", "url": "https://wpnews.pro/news/medqa-mm-shortcuts-behind-medical-visual-reasoning", "canonical_source": "https://arxiv.org/abs/2609.03261", "published_at": "2026-09-04 04:00:00+00:00", "updated_at": "2026-09-04 04:24:40.832029+00:00", "lang": "en", "topics": ["artificial-intelligence", "machine-learning", "ai-research", "ai-ethics"], "entities": ["arXiv", "MedQA-MM"], "alternates": {"html": "https://wpnews.pro/news/medqa-mm-shortcuts-behind-medical-visual-reasoning", "markdown": "https://wpnews.pro/news/medqa-mm-shortcuts-behind-medical-visual-reasoning.md", "text": "https://wpnews.pro/news/medqa-mm-shortcuts-behind-medical-visual-reasoning.txt", "jsonld": "https://wpnews.pro/news/medqa-mm-shortcuts-behind-medical-visual-reasoning.jsonld"}}