{"slug": "pgpo-potential-guided-policy-optimization-for-multi-turn-agentic-tasks", "title": "PGPO: Potential-Guided Policy Optimization for Multi-Turn Agentic Tasks", "summary": "Researchers propose Potential-Guided Policy Optimization (PGPO), a new reinforcement learning method for multi-turn agentic tasks that estimates empirical state potentials from anchor-state-group return statistics to derive action advantages from potential differences between adjacent states, enabling cross-trajectory credit propagation. Experiments on ALFWorld and WebShop show strong overall performance relative to recent group-based RL methods, with more informative failure-side credit signals and negligible training overhead.", "body_md": "arXiv:2609.02236v1 Announce Type: new\nAbstract: Group-based reinforcement learning (RL) has become an effective paradigm for LLM post-training, but in multi-turn agentic tasks with sparse terminal rewards, it often provides coarse credit for intermediate actions. To obtain more fine-grained credit assignment, recent work such as GiGPO introduces step-level advantages for intermediate actions. However, these step-level signals still rely on the final outcome of each individual trajectory. As a result, actions within failed trajectories can remain poorly differentiated, so effective actions can receive the same unfavorable credit as erroneous ones. In this work, we propose Potential-Guided Policy Optimization (PGPO) for multi-turn agentic tasks. PGPO estimates empirical state potentials from anchor-state-group return statistics within each rollout group. It then derives action advantages from potential differences between adjacent states, enabling cross-trajectory credit propagation. This provides finer-grained step-level credit assignment, especially within failed trajectories. Experiments on ALFWorld and WebShop show strong overall performance relative to recent group-based RL methods. Further analysis provides evidence that PGPO yields more informative failure-side credit signals with negligible training overhead.", "url": "https://wpnews.pro/news/pgpo-potential-guided-policy-optimization-for-multi-turn-agentic-tasks", "canonical_source": "https://www.machinebrief.com/news/pgpo-potential-guided-policy-optimization-for-multi-turn-age-301x", "published_at": "2026-09-03 04:00:00+00:00", "updated_at": "2026-09-03 06:22:13.104832+00:00", "lang": "en", "topics": ["artificial-intelligence", "machine-learning"], "entities": ["PGPO", "GiGPO", "ALFWorld", "WebShop"], "alternates": {"html": "https://wpnews.pro/news/pgpo-potential-guided-policy-optimization-for-multi-turn-agentic-tasks", "markdown": "https://wpnews.pro/news/pgpo-potential-guided-policy-optimization-for-multi-turn-agentic-tasks.md", "text": "https://wpnews.pro/news/pgpo-potential-guided-policy-optimization-for-multi-turn-agentic-tasks.txt", "jsonld": "https://wpnews.pro/news/pgpo-potential-guided-policy-optimization-for-multi-turn-agentic-tasks.jsonld"}}