{"slug": "agent-explorative-policy-optimization-for-multimodal-agentic-reasoning", "title": "Agent Explorative Policy Optimization for Multimodal Agentic Reasoning", "summary": "Researchers propose AXPO (Agent eXplorative Policy Optimization), a reinforcement learning method that addresses the Thinking-Acting Gap in multimodal agentic reasoning by resampling tool calls for all-wrong subgroups. Across nine multimodal benchmarks and three scales of Qwen3-VL-Thinking, SFT+AXPO outperforms SFT+GRPO by +1.8pp Pass@1 and +1.8pp Pass@4 on average at 8B, and the 8B model surpasses the 32B Base on Pass@4 with 4 times fewer parameters.", "body_md": "Vision-language models with extended reasoning succeed on complex problems, but many real-world problems require external tools that internal reasoning alone often cannot resolve. Agentic reasoning therefore interleaves two behaviors with a structural asymmetry: thinking (the self-contained default) and tool use (a high-variance auxiliary acting). We refer to this asymmetry as the Thinking-Acting Gap. Under standard RL recipes like GRPO, the gap manifests as two diagnostic symptoms during training: tool use is attempted on only ~30% of rollouts, and when attempted, the tool-using rollouts within a group are all-wrong on ~40% of questions, suppressing the learning signal at the tool calls that needed it. We propose AXPO (Agent eXplorative Policy Optimization): for each all-wrong tool-using subgroup, AXPO fixes the thinking prefix and resamples the tool call and its continuation, paired with uncertainty-based prefix selection. Across nine multimodal benchmarks and three scales of Qwen3-VL-Thinking, SFT+AXPO outperforms SFT+GRPO at average (+1.8pp Pass@1 and +1.8pp Pass@4 at 8B on average) and 8B with SFT+AXPO surpasses the 32B Base on Pass@4 with 4 times fewer parameters.", "url": "https://wpnews.pro/news/agent-explorative-policy-optimization-for-multimodal-agentic-reasoning", "canonical_source": "https://research.nvidia.com/publication/2026-12_agent-explorative-policy-optimization-multimodal-agentic-reasoning", "published_at": "2026-08-11 07:55:56+00:00", "updated_at": "2026-08-11 08:08:37.602019+00:00", "lang": "en", "topics": ["artificial-intelligence", "machine-learning"], "entities": ["AXPO", "GRPO", "Qwen3-VL-Thinking"], "alternates": {"html": "https://wpnews.pro/news/agent-explorative-policy-optimization-for-multimodal-agentic-reasoning", "markdown": "https://wpnews.pro/news/agent-explorative-policy-optimization-for-multimodal-agentic-reasoning.md", "text": "https://wpnews.pro/news/agent-explorative-policy-optimization-for-multimodal-agentic-reasoning.txt", "jsonld": "https://wpnews.pro/news/agent-explorative-policy-optimization-for-multimodal-agentic-reasoning.jsonld"}}