OPD

Researchers propose ATOD, a hybrid online distillation algorithm that combines on-policy distillation and reinforcement learning to train small language-model agents for multi-turn tasks. ATOD uses an…

Researchers have introduced On-Policy Representation Distillation (OPRD), a method that aligns student and teacher model representations across selected layers during training, bypassing the language …