GraphVerse: A Comprehensive Visual Graph Reasoning Benchmark for Multimodal Large Language Models

Researchers introduced GraphVerse, a benchmark for evaluating multimodal large language models (MLLMs) on visual graph reasoning, which tests perception, structural understanding, and multi-step reasoning over graph-based visual inputs in both single-image and paired-image settings. The benchmark includes Graph-centric Image Editing (GIE) strategies and a process-sensitive metric called VGR-Score, and experiments revealed limitations of current MLLMs in visual graph reasoning. The code is available at https://github.com/sunyuanfu/GraphVerse.

arXiv:2608.06769v1 Announce Type: new Abstract: Recent Multimodal Large Language Models MLLMs have achieved remarkable progress across diverse vision-language tasks, creating an urgent need for more challenging benchmarks. Yet existing evaluations still provide limited insight into whether these models can truly reason over structured visual information. Visual Graph Reasoning VGR offers a compelling testbed for this challenge, requiring models to integrate perception, structural understanding, and multi-step reasoning over graph-based visual inputs. However, prior VGR benchmarks often reduce the task to visual perception followed by text-based reasoning, restrict evaluation to single-image settings, rely on answer-only metrics, and underrepresent realistic graph-centric scenarios. To bridge the gap, we introduce GraphVerse, a unified benchmark that jointly evaluates perception, visual reasoning, and text-based graph reasoning in MLLMs under both single-image and paired-image settings. At its core is a suite of Graph-centric Image Editing GIE strategies that modify graph images while preserving their semantics, turning them into active tests of visual reasoning. We further propose VGR-Score, a process-sensitive metric that evaluates reasoning quality beyond final-answer accuracy. Extensive experiments reveal several key limitations of current MLLMs in VGR, while also validating the effectiveness of GIE strategies and the transferability of GraphVerse to broader multimodal reasoning capabilities. The code is available at https://github.com/sunyuanfu/GraphVerse.