EvaOmni: An Evidence-Grounded Benchmark for Audio-Visual Reasoning

A benchmark named EvaOmni targets audio-visual reasoning in omni LLMs by requiring models to locate key audio and visual evidence and integrate information across modalities, rather than scoring only answer correctness as existing multiple-choice audio-visual benchmarks do. The benchmark's stated aim is to address the limits of answer-only scoring, which the authors say makes it difficult to evaluate how models reach their conclusions.

Omni LLMs can naturally locate key audio and visual evidence and integrate information across modalities for audiovisual reasoning. Existing audio-visual reasoning benchmarks that mainly rely on multiple-choice questions usually score models solely based on answer correctness, making it difficult to