Beyond Prediction: Steering VLM Agents with Retrospective World Modeling

A new arXiv paper (arXiv:2609.39101v1) introduces Retrospective World Modeling, an agent learning paradigm that estimates the retrospective attribution distribution P(â_t|s_t, s_{t+1}) to identify the action most likely to have caused an observed state transition. The authors pair this with a Self-Consistency Reward (SCR), an intrinsic signal measuring probabilistic consistency between the policy action and the retrospective explanation, integrated into reinforcement learning to provide dense transition-level feedback. Experiments across diverse agentic tasks show the method improves policy robustness and generalization over prospective-only world modeling baselines.

arXiv:2609.39101v1 Announce Type: new Abstract: Equipping VLM agents with world modeling capabilities has shown strong potential for complex reasoning and long-horizon planning, while reducing the dependence of policy learning on costly real-world interactions. Existing methods mainly rely on prospective simulation to predict the consequences of candidate actions. However, this forward-only paradigm focuses on what will happen next and provides limited constraints for verifying whether an action is causally consistent with the observed state transition, which can lead to plausible-looking but physically incoherent behaviors. In this paper, we challenge the view of world modeling as only prospective prediction and introduce Retrospective World Modeling, a new agent learning paradigm that enables agents to reason backward by estimating the retrospective attribution distribution $P \hat{a}{t}|s t, s{t+1} $ for the action that most likely caused a given transition. Based on this capability, we formulate the Self-Consistency Reward SCR , an intrinsic signal that measures the probabilistic consistency between the policy action and the retrospective explanation. Integrating SCR into reinforcement learning provides dense transition-level feedback and steers agents toward behaviors that are both task-effective and physically grounded. Extensive experiments across diverse agentic tasks show that our method substantially improves policy robustness and generalization over prospective-only world modeling baselines.