{"slug": "which-successes-make-good-references-action-sequence-alignment-for-multi-turn-on", "title": "Which Successes Make Good References? Action-Sequence Alignment for Multi-Turn On-Policy Distillation", "summary": "A new arXiv paper (2611.13135v1) proposes selecting successful reference trajectories for multi-turn on-policy distillation by action-sequence alignment, using longest common subsequence (LCS) as an observable proxy for trajectory compatibility. The authors cast successful-reference selection as a free-target optimal transport problem that reduces to assigning each failed trajectory its most compatible successful reference. Experiments on ALFWorld and the airline domain of τ³-bench show LCS is the only evaluated reference selector to improve over reference-free OPD on both ALFWorld overall scores and all three airline metrics, and airline-trained models transferred to the retail domain with LCS improving all three metrics.", "body_md": "arXiv:2611.13135v1 Announce Type: new \nAbstract: On-policy distillation (OPD) reduces train-inference distribution mismatch by supervising a student on its own trajectories. For multi-turn tool-use agents, conditioning the teacher on a validated same-task successful trajectory as a reference can provide useful hindsight, but different successes may reach incompatible interaction states and offer misleading supervision. We cast successful-reference selection as a free-target optimal transport problem, which reduces to assigning each failed trajectory its most compatible successful reference. Because latent state paths are hidden, we use action-sequence alignment as an observable proxy for compatibility and instantiate it with longest common subsequence (LCS), which identifies a shared ordered backbone of actions while allowing recoverable detours. This perspective makes trajectory compatibility the central criterion for reference selection in multi-turn OPD. Experiments on ALFWorld and the airline domain of $\\tau^3$-bench show that LCS is the only evaluated reference selector to improve over reference-free OPD on both ALFWorld overall scores and all three airline metrics. Without further optimization, the airline-trained models also transfer to the retail domain, where LCS improves all three metrics.", "url": "https://wpnews.pro/news/which-successes-make-good-references-action-sequence-alignment-for-multi-turn-on", "canonical_source": "https://arxiv.org/abs/2611.13135", "published_at": "2026-11-11 05:00:00+00:00", "updated_at": "2026-11-11 05:22:15.794017+00:00", "lang": "en", "topics": ["artificial-intelligence", "machine-learning", "ai-agents", "ai-research", "large-language-models"], "entities": ["arXiv", "ALFWorld", "τ³-bench"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/which-successes-make-good-references-action-sequence-alignment-for-multi-turn-on", "markdown": "https://wpnews.pro/news/which-successes-make-good-references-action-sequence-alignment-for-multi-turn-on.md", "text": "https://wpnews.pro/news/which-successes-make-good-references-action-sequence-alignment-for-multi-turn-on.txt", "jsonld": "https://wpnews.pro/news/which-successes-make-good-references-action-sequence-alignment-for-multi-turn-on.jsonld"}}