A critical review of on-policy self-distillation for language models finds that the method, which trains a model on its own generations with token-level teacher scores, requires a second, larger teacher model, limiting its practicality. The review highlights the trade-offs between dense supervision and on-policy sampling inherent in the approach.
On-policy distillation trains a language model on its own generations while a teacher scores them token by token. It combines the dense supervision of imitation learning with the on-policy sampling of reinforcement learning. But it requires a second, larger model to act as teacher. On-Policy Self-Di
EU-based hosting, git-push deploys, automatic HTTPS, no cold starts. Free tier with a custom domain — perfect for shipping the agent you just read about.