Breaking the Tokenizer Barrier: On-Policy Distillation Across Model Families
Researchers have developed a method for on-policy distillation (OPD) that works across different tokenizers, enabling knowledge transfer between large language models (LLMs) from different model families. The approach us…