Retroactive Advantage Correction: Closed-Form V-Trace Bias Correction for Delay-Aware RLHF
Researchers introduced Retroactive Advantage Correction (RAC), a method for reinforcement learning from human feedback (RLHF) that handles delayed reward signals. RAC reduces policy bias by up to 47.9x in tabular MDPs co…