Researchers propose RSTG (Recovering Learning Signals via Adaptive Teacher Guidance), a method that selectively applies on-policy distillation to negative zero-variance prompts in reinforcement learni…
Researchers propose SAF, a Stable Advantage Fusion framework that combines reinforcement learning with verifiable rewards (RLVR) and on-policy distillation (OPD) for training language models, addressi…
Researchers propose ATOD, a hybrid online distillation algorithm that combines on-policy distillation and reinforcement learning to train small language-model agents for multi-turn tasks. ATOD uses an…
Researchers have introduced On-Policy Representation Distillation (OPRD), a method that aligns student and teacher model representations across selected layers during training, bypassing the language …