ProAS: A Progress-Aware Advantage Shaping Framework for Multi-Turn Agentic Reinforcement Learning

A paper titled "ProAS: A Progress-Aware Advantage Shaping Framework for Multi-Turn Agentic Reinforcement Learning" proposes progress-aware advantage shaping to address what its authors call the uniform credit assignment dilemma in critic-free reinforcement learning algorithms such as GRPO, which broadcast trajectory-level advantages equally to all actions when optimizing Large Language Model agents. The work targets multi-turn agentic reinforcement learning, where RL has become the primary paradigm for optimizing LLM agents.

Reinforcement learning RL has emerged as the primary paradigm for optimizing Large Language Model LLM agents, yet efficient critic-free algorithms e.g., GRPO suffer from a fundamental uniform credit assignment dilemma that broadcasts trajectory-level advantages equally to all actions. Our anal