{"slug": "reflectpo-reflection-guided-policy-optimization-for-long-horizon-llm-agents", "title": "ReflectPO: Reflection-Guided Policy Optimization for Long-Horizon LLM Agents", "summary": "A new arXiv paper (2610.30933v1) proposes Reflection-Guided Policy Optimization (ReflectPO), a reinforcement learning framework that lets LLM-based agents recover autonomously from errors in long-horizon tasks. ReflectPO combines Reflection-Guided Sampling, which uses branch differences and repeated visits in interaction trajectories to build error-recovery hints, with Tree-Structured Credit Assignment, which organizes sampled trajectories into trees and assigns action-level credit from parent-child reward differences and recovery gains. Across two long-horizon tool-use benchmarks, ReflectPO outperforms multiple existing baselines in most domains and posts the best overall average, with further analyses showing improved agent recognition of deviations and recovery from erroneous states.", "body_md": "arXiv:2610.30933v1 Announce Type: new \nAbstract: With external tools, large language models (LLMs) can tackle complex tasks through sustained environmental interaction. LLM-based agents are therefore increasingly used to tackle complex long-horizon tasks in real-world environments. However, these tasks involve sequential, interdependent decisions and changes in environmental states. Early execution errors affect subsequent interaction histories and environmental states, and their effects accumulate along the decision chain. To address this challenge, we propose Reflection-Guided Policy Optimization (ReflectPO), a reinforcement learning framework for autonomous error recovery in long-horizon tasks. ReflectPO comprises two core components. The first is Reflection-Guided Sampling. We use branch differences and repeated visits in interaction trajectories to construct hints for error recovery. The second is Tree-Structured Credit Assignment. We organize sampled trajectories into trees and assign action-level credit using parent--child reward differences and recovery gains. Experiments over two long-horizon tool-use benchmarks show that ReflectPO outperforms multiple existing baselines in most domains. It also achieves the best overall performance on average. Further analyses show that ReflectPO effectively improves agents' ability to recognize their own deviations, recover from erroneous states, and continue making progress toward task completion.", "url": "https://wpnews.pro/news/reflectpo-reflection-guided-policy-optimization-for-long-horizon-llm-agents", "canonical_source": "https://www.machinebrief.com/news/reflectpo-reflection-guided-policy-optimization-for-long-hor-zks6", "published_at": "2026-10-23 04:00:00+00:00", "updated_at": "2026-10-23 04:42:49.833366+00:00", "lang": "en", "topics": ["ai-agents", "ai-research", "large-language-models", "machine-learning", "ai-tools"], "entities": ["ReflectPO", "Reflection-Guided Policy Optimization", "Reflection-Guided Sampling", "Tree-Structured Credit Assignment", "arXiv"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/reflectpo-reflection-guided-policy-optimization-for-long-horizon-llm-agents", "markdown": "https://wpnews.pro/news/reflectpo-reflection-guided-policy-optimization-for-long-horizon-llm-agents.md", "text": "https://wpnews.pro/news/reflectpo-reflection-guided-policy-optimization-for-long-horizon-llm-agents.txt", "jsonld": "https://wpnews.pro/news/reflectpo-reflection-guided-policy-optimization-for-long-horizon-llm-agents.jsonld"}}