# How Much Training-Inference Mismatch Can On-Policy Distillation Tolerate? > Source: > Published: 2026-11-10 09:00:01+00:00 On-policy distillation (OPD) trains a student language model on its own responses, guided by a fixed teacher. Generating these responses is expensive, so systems use quantized inference and asynchronous rollouts to reduce the cost. Both, however, introduce training-inference mismatch between the rol