{"slug": "omnimapbench-benchmarking-visual-centric-reasoning-on-diverse-map-documents", "title": "OmniMapBench: Benchmarking Visual-Centric Reasoning on Diverse Map Documents", "summary": "Researchers introduced OmniMapBench, a benchmark of 2,096 question-answer pairs across 1,603 map documents designed to test visual-centric reasoning in large vision-language models. The benchmark uses a Visual Dependency Index to measure reliance on visual information, and evaluations of 25 leading models found the top performer achieved only 75.03% accuracy, highlighting significant challenges in current AI systems.", "body_md": "arXiv:2607.09068v1 Announce Type: new\nAbstract: Recent advancements in LVLMs necessitate robust benchmarks for complex, visually grounded reasoning. A critical limitation is identified in many document understanding benchmarks: visual content is often reducible to text, enabling high performance without genuine visual grounding. To address this limitation, OmniMapBench is introduced to foster visual-centric reasoning for map documents. The benchmark comprises 2,096 manually annotated question-answer pairs across 1,603 map documents from nine categories. It is designed to probe a hierarchy of skills, ranging from perception to multi-step visual reasoning. To quantify benchmark properties, a simple yet effective benchmark-level metric is proposed: the Visual Dependency Index (VDI), defined as the accuracy drop when images are replaced with question-agnostic descriptions. OmniMapBench exhibits higher VDI than established benchmarks, which quantitatively validates its focus on irreducible visual reasoning. Comprehensive evaluations of 25 leading LVLMs are conducted on OmniMapBench. A significant performance gap is observed, with the top-performing model achieving only 75.03\\% accuracy. This result underscores the challenges posed by OmniMapBench to current LVLMs. This work aims to catalyze progress in visual-centric reasoning for document understanding of LVLMs. The dataset and code are publicly available at https://github.com/SIGMME/OmniMapBench.", "url": "https://wpnews.pro/news/omnimapbench-benchmarking-visual-centric-reasoning-on-diverse-map-documents", "canonical_source": "https://arxiv.org/abs/2607.09068", "published_at": "2026-07-13 04:00:00+00:00", "updated_at": "2026-07-13 04:25:34.228127+00:00", "lang": "en", "topics": ["artificial-intelligence", "computer-vision", "natural-language-processing", "ai-research", "ai-tools"], "entities": ["OmniMapBench", "SIGMME"], "alternates": {"html": "https://wpnews.pro/news/omnimapbench-benchmarking-visual-centric-reasoning-on-diverse-map-documents", "markdown": "https://wpnews.pro/news/omnimapbench-benchmarking-visual-centric-reasoning-on-diverse-map-documents.md", "text": "https://wpnews.pro/news/omnimapbench-benchmarking-visual-centric-reasoning-on-diverse-map-documents.txt", "jsonld": "https://wpnews.pro/news/omnimapbench-benchmarking-visual-centric-reasoning-on-diverse-map-documents.jsonld"}}