HarnessRL: Co-Evolving Policies with a Just-in-Time Training Harness for Open-Ended RL

A research paper titled "HarnessRL: Co-Evolving Policies with a Just-in-Time Training Harness for Open-Ended RL" identifies two failure modes in rubric-based reinforcement learning for open-ended generation, where natural-language criteria serve as rewards: under non-discriminative rewards, scores collapse to uniformly low or high values as training progresses. The work proposes co-evolving policies with a just-in-time training harness to address the degradation of rubric-based rewards.

Rubric-based reinforcement learning RL trains policies for open-ended generation with natural-language criteria as rewards, but these rewards can become ineffective as training progresses. We identify two failure modes. Under non-discriminative rewards, scores collapse to uniformly low or high val