17:00
2026-09-01
dev.to
machine-learning
On-Policy Distillation Works Better Without the Teacher
Researchers at Purdue University, Yi Ding and Ruqi Zhang, found that teacher supervision in on-policy distillation is largely noise, and larger teacher models produce more noise. They introduced On-Po…