On-policy learning has been argued to reduce catastrophic forgetting, produce sparser parameter updates, and improve generalisation. However, existing comparisons between supervised fine-tuning and reinforcement learning vary many factors simultaneously, making the contribution of rollout policy dif
On-Policy or Off-Policy Learning? A Systematic Study of Distillation Dynamics
A systematic study of distillation dynamics finds that existing comparisons between supervised fine-tuning and reinforcement learning vary too many factors at once, making it impossible to isolate the contribution of rollout policy to on-policy learning's claimed benefits of reduced catastrophic forgetting, sparser parameter updates, and improved generalisation. The research examines on-policy versus off-policy learning to disentangle those factors.
Prior supervised fine-tuning versus reinforcement learning comparisons conflate multiple variables, so the effect of rollout policy on forgetting, parameter sparsity and generalisation cannot be isolated from existing results.
Run your AI side-project on zahid.host
EU-based hosting, git-push deploys, automatic HTTPS, no cold starts. Free tier with a custom domain — perfect for shipping the agent you just read about.