Beyond Teacher Assignment: Domain-Normalized Multi-Teacher On-Policy Distillation

Multi-teacher on-policy distillation (MOPD) merges specialist language models trained via reinforcement learning into a single model, but the technique suffers from a domain-imbalance problem that the paper addresses with a domain-normalized variant. The work targets the gap between single-skill specialists in mathematics, coding and instruction following and users' need for one model with all of these skills.

Reinforcement learning can turn one language model into several specialists, each excellent at a single skill such as mathematics, coding or following instructions, but users need one model with all of these skills. Multi-teacher on-policy distillation MOPD merges them by letting the specialists t