{"slug": "triad-benchmarking-omni-modal-ambiguity-in-multimodal-large-language-models", "title": "TRIAD: Benchmarking Omni-Modal Ambiguity in Multimodal Large Language Models", "summary": "A new benchmark, TRIAD, finds that the best of 14 omni-modal large language models trails humans by more than 20 points in scene-level group accuracy on ambiguity resolution, even though item-level scores look strong. TRIAD, described in arXiv paper 2610.30841v1, contains 327 hand-written text-image-audio queries in 86 scene groups spanning 18 ambiguity categories, each built so the full triplet yields a unique answer while dropping any one modality leaves multiple consistent options. Removing answer options pushes model group accuracy into the single digits while humans stay robust, and leave-one-out and transcript/description counterfactuals show current systems often underuse audio cues when text and image are present.", "body_md": "arXiv:2610.30841v1 Announce Type: new \nAbstract: Ambiguity is a central feature of natural communication: a sentence, image, or sound may remain underspecified until evidence from the other channels is considered. Yet current multimodal benchmarks mostly test perception, recognition, or reasoning over already determinate inputs, leaving unclear whether omni-modal large language models can resolve ambiguity distributed across text, vision, and audio. We introduce TRIAD, a benchmark for omni-modal ambiguity resolution. Each item is a text--image--audio triplet paired with a question and a set of answer options, and is constructed so that the full triplet determines a unique answer while removing any one modality makes the set of consistent options non-unique. TRIAD contains $327$ hand-written queries in $86$ scene groups and covers $18$ ambiguity categories across the three modalities. Evaluating $14$ omni-modal systems reveals that strong item-level performance does not translate into robust scene-level disambiguation: the best model still trails humans by over $20$ points in group accuracy. Removing answer options widens the gap further, driving model group accuracy to the single digits while humans remain robust. Leave-one-out and transcript/description counterfactuals further show that current systems often underuse audio cues when text and image are present. TRIAD reframes omni-modal evaluation around disambiguation rather than recognition, and provides a taxonomy, benchmark, and diagnostic protocol for measuring this capability.", "url": "https://wpnews.pro/news/triad-benchmarking-omni-modal-ambiguity-in-multimodal-large-language-models", "canonical_source": "https://www.machinebrief.com/news/triad-benchmarking-omni-modal-ambiguity-in-multimodal-large-zk10", "published_at": "2026-10-23 04:00:00+00:00", "updated_at": "2026-10-23 04:42:28.126263+00:00", "lang": "en", "topics": ["artificial-intelligence", "large-language-models", "ai-research", "machine-learning", "natural-language-processing"], "entities": ["TRIAD", "arXiv"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/triad-benchmarking-omni-modal-ambiguity-in-multimodal-large-language-models", "markdown": "https://wpnews.pro/news/triad-benchmarking-omni-modal-ambiguity-in-multimodal-large-language-models.md", "text": "https://wpnews.pro/news/triad-benchmarking-omni-modal-ambiguity-in-multimodal-large-language-models.txt", "jsonld": "https://wpnews.pro/news/triad-benchmarking-omni-modal-ambiguity-in-multimodal-large-language-models.jsonld"}}