When Implausible Tokens Get Reinforced: Tail-Aware Credit Calibration for LLM Reinforcement Learning
Researchers identified a failure mode in reinforcement learning for large language models called Positive-Credit Contamination, where low-probability tail tokens receive undeserved positive credit. They propose TACO, a m…