Surprising Success, Repeated Failure: Entropy-Guided Credit Assignment for Exploration in LLM Reasoning

A new reinforcement learning method called Entropy-Guided Credit Assignment (EGCA) improves exploration in large language model reasoning by assigning credit at a finer granularity without auxiliary models, additional sampling, or privileged information, according to the paper. The approach targets RLVR, which enhances LLM reasoning through outcome-level feedback, and addresses the "surprising success, repeated failure" pattern in which models succeed on hard problems while repeatedly failing on easier ones. The work claims finer-grained credit assignment can be achieved using policy entropy alone.

Reinforcement learning with verifiable rewards RLVR enhances reasoning in large language models LLMs through outcome-level feedback, yet recent approaches to finer-grained credit assignment often require auxiliary models, additional sampling, or privileged information. Although policy entropy pr