{"slug": "behavior-leverage-imbalance-in-multi-teacher-on-policy-distillation", "title": "Behavior Leverage Imbalance in Multi-Teacher On-Policy Distillation", "summary": "Researchers at an undisclosed institution found that multi-teacher on-policy distillation for agentic language models can cause over-calling of tools, where models call tools on examples that should be answered directly. They propose Soft Clamp, a per-token divergence calibration method that reduces over-calling from 13.7% to 9.0% on APIGen-MT while maintaining decision accuracy. The work highlights the need to monitor where teacher signals act, not just aggregate loss, in multi-teacher distillation.", "body_md": "arXiv:2607.07050v1 Announce Type: new\nAbstract: Agentic language models must learn when to call tools, when to consume tool responses, and when to answer directly. This makes multi-teacher on-policy distillation a natural training strategy: one teacher can specialize in tool calls, another in direct responses, and the student can learn from both on\nits own generated distribution. We show that this strategy can induce a behavior shift that is invisible from aggregate losses alone. In a two-teacher tool-use setting, vanilla generalized knowledge distillation improves tool-call recall but also moves the model toward over-calling, where it calls tools\non examples that should be answered directly. Aggregate explanations are insufficient: tool-call samples do not receive more token exposure, and full-sequence per-token divergence is not larger for the tool-call teacher. We instead analyze behavior leverage imbalance: local token-level signals at mode-\nentry and structural positions, such as and function names, can have disproportionate control over the global generation mode. We propose Soft Clamp, a per-token divergence calibration method that dynamically compresses extreme token-level Jensen-Shannon divergence while preserving nonzero\ngradients. On APIGen-MT, Soft Clamp reduces over-calling from 13.7% to 9.0% relative to vanilla GKD while matching its decision accuracy. In a BFCL multi-turn diagnostic, it also lowers tool-call loops and repeated calls among GKD variants. These results suggest that multi-teacher OPD should monitor\nwhere teacher signals act, not only how large they are in aggregate.", "url": "https://wpnews.pro/news/behavior-leverage-imbalance-in-multi-teacher-on-policy-distillation", "canonical_source": "https://arxiv.org/abs/2607.07050", "published_at": "2026-07-09 04:00:00+00:00", "updated_at": "2026-07-09 04:15:23.043090+00:00", "lang": "en", "topics": ["large-language-models", "ai-agents", "machine-learning", "ai-research"], "entities": ["APIGen-MT", "BFCL", "Soft Clamp"], "alternates": {"html": "https://wpnews.pro/news/behavior-leverage-imbalance-in-multi-teacher-on-policy-distillation", "markdown": "https://wpnews.pro/news/behavior-leverage-imbalance-in-multi-teacher-on-policy-distillation.md", "text": "https://wpnews.pro/news/behavior-leverage-imbalance-in-multi-teacher-on-policy-distillation.txt", "jsonld": "https://wpnews.pro/news/behavior-leverage-imbalance-in-multi-teacher-on-policy-distillation.jsonld"}}