IVR-R1: Refining Trajectories through Iterative Visual-Grounded Reasoning in Reinforcement Learning

Researchers have introduced IVR-R1, a reinforcement learning framework that iteratively realigns visual reasoning trajectories to correct errors in multimodal large language models. The method uses a reward-driven screening mechanism to identify flawed reasoning steps and a Re-Reasoning Loop that cross-references intermediate states against original visual data to synthesize expert-level demonstrations. In tests across multiple benchmarks, IVR-R1 outperformed existing reinforcement learning approaches by maintaining logical and visual consistency in complex, long-horizon reasoning tasks.

arXiv:2605.23997v1 Announce Type: new Abstract: Multimodal large language models via reinforcement learning RL have demonstrated remarkable capabilities in complex visual reasoning tasks, yet they remain limited in long-horizon multimodal scenarios, often suffering from visual hallucination and logical error. Current methods typically pre-encode high-dimensional visual scenes into discrete textual proxies to facilitate downstream reasoning. As the reasoning chain unfolds, however, the inherent information asymmetry between text and visual scenes tends to erode visual grounding, resulting in misguided reasoning and erroneous outputs. To address this issue, we introduce IVR-R1 Iterative Visual-grounded Reasoning , a novel RL training framework that facilitates dynamic visual re-alignment that actively rectifies reasoning trajectories to guide policy optimization. Specifically, by leveraging a reward-driven screening mechanism to identify flawed rollouts, IVR-R1 executes a fine-grained, step-level error attribution within the multimodal context. By iteratively cross-referencing intermediate reasoning states against pristine visual priors, a Re-Reasoning Loop enables automated trajectory rectification, effectively synthesizing expert-level demonstrations that serve as high-fidelity reasoning templates for the policy model. Our experiments across diverse multimodal benchmarks demonstrate that IVR-R1 consistently outperforms existing reinforcement learning methods, establishing a superior paradigm for maintaining logical and visual consistency in complex multimodal reasoning.