Group Adaptive Clipping Policy Optimization

Researchers propose Group Adaptive Clipping Policy Optimization to address limitations in group relative policy optimization for reinforcement learning with verifiable rewards, which uses a fixed importance-sampling ratio clipping boundary across all rollouts. The method adapts clipping based on problem difficulty, improving performance on harder problems.

Group relative policy optimization for reinforcement learning with verifiable rewards RLVR typically uses a fixed importance-sampling IS ratio clipping boundary across all rollouts. We identify a key limitation: rare correct rollouts on harder problems and abundant correct rollouts on easier pro