A new analysis examines how much training-inference mismatch on-policy distillation (OPD) can tolerate, given that OPD trains a student language model on its own responses under a fixed teacher and that quantized inference and asynchronous rollouts—used to cut the cost of generating those responses—both introduce mismatch between the rollout and training. The work frames the tradeoff between cheaper rollout generation and the divergence it creates between the inference and training distributions.
On-policy distillation (OPD) trains a student language model on its own responses, guided by a fixed teacher. Generating these responses is expensive, so systems use quantized inference and asynchronous rollouts to reduce the cost. Both, however, introduce training-inference mismatch between the rol
EU-based hosting, git-push deploys, automatic HTTPS, no cold starts. Free tier with a custom domain — perfect for shipping the agent you just read about.