A research paper proposes learning from teachable prefixes to decide when a teacher model should intervene in On-Policy Self-Distillation (OPSD) for multilingual reasoning distillation. The work addresses the problem that OPSD's teacher, the same model conditioned on reference solutions, must give useful feedback across languages even when the student's reasoning is flawed. The paper's framing targets the timing of teacher feedback rather than applying distillation uniformly across languages.
On-Policy Self-Distillation (OPSD) improves reasoning by training a student on its own reasoning, using the same model conditioned on reference solutions as a teacher. Applying OPSD to multilingual reasoning requires the teacher to provide useful feedback across languages, even when the student make
EU-based hosting, git-push deploys, automatic HTTPS, no cold starts. Free tier with a custom domain — perfect for shipping the agent you just read about.