{"slug": "one-model-two-paths-balancing-single-pass-and-looped-performance-in-language", "title": "One Model, Two Paths: Balancing Single-Pass and Looped Performance in Language Models", "summary": "A new arXiv paper (2610.33517v1) reports that jointly training a pretrained Transformer with weighted losses on both single-pass and looped outputs substantially reduces single-pass degradation compared to looped-only training while preserving looped gains over ordinary continued training. The authors find most of the benefit comes from adjusting the single-pass loss weight, with guidance from the frozen original model further improving the looped output, and that validation-selected weights keep single-pass quality within a predefined tolerance of ordinary continued training. The work addresses the trade-off created when a model reuses middle-layer weights for extra computation, since shared weights mean training for looped predictions can hurt single-pass quality.", "body_md": "arXiv:2610.33517v1 Announce Type: new \nAbstract: Repeating Transformer layers can improve language-model performance without adding parameters. A pretrained model can be adapted to support both single-pass and looped inference by reusing middle-layer weights for additional computation. Because both paths share weights, training to improve looped predictions can degrade single-pass quality. We study joint training strategies using weighted losses on both outputs, with guidance from the frozen original model. Joint supervision substantially reduces single-pass degradation compared to looped-only training, while preserving looped gains over ordinary continued training. Most of this benefit comes from adjusting the single-pass loss weight, with guidance from the frozen original model further improving the looped output. With validation-selected weights, our method preserves single-pass quality within a predefined tolerance of ordinary continued training while improving looped predictions.", "url": "https://wpnews.pro/news/one-model-two-paths-balancing-single-pass-and-looped-performance-in-language", "canonical_source": "https://www.machinebrief.com/news/one-model-two-paths-balancing-single-pass-and-looped-perform-rowy", "published_at": "2026-10-27 04:00:00+00:00", "updated_at": "2026-10-27 04:42:20.517912+00:00", "lang": "en", "topics": ["large-language-models", "machine-learning", "ai-research", "natural-language-processing"], "entities": ["arXiv", "Transformer"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/one-model-two-paths-balancing-single-pass-and-looped-performance-in-language", "markdown": "https://wpnews.pro/news/one-model-two-paths-balancing-single-pass-and-looped-performance-in-language.md", "text": "https://wpnews.pro/news/one-model-two-paths-balancing-single-pass-and-looped-performance-in-language.txt", "jsonld": "https://wpnews.pro/news/one-model-two-paths-balancing-single-pass-and-looped-performance-in-language.jsonld"}}