{"slug": "decoding-looped-transformers-better-for-almost-free", "title": "Decoding Looped Transformers Better for Almost Free", "summary": "Researchers posted a paper on arXiv on 1 October 2026 introducing LoopCD, a training-free contrastive decoding framework for looped Transformers that contrasts the final prediction with an earlier recurrent pass. LoopCD-Logits raised Ouro-2.6B-Thinking's AIME 2024 pass@1 from 61.88% to 73.33%, and LoopCD-Hidden lifted Huginn's HumanEval pass@1 from 22.56% to 31.71%, while halving recurrent loops still matched or exceeded full-depth unguided baselines and cut forward FLOPs by 22.5% to 48.2%.", "body_md": "# Computer Science > Machine Learning\n\n  [Submitted on 1 Oct 2026]\n\n# Title:Decoding Looped Transformers Better for (Almost) Free\n\n[View PDF](https://arxiv.org/pdf/2610.02185)\n\n[HTML (experimental)](https://arxiv.org/html/2610.02185v1)\n\nAbstract:Looped Transformers achieve parameter efficiency by repeatedly executing a shared block across recurrent loops. Each loop yields an intermediate representation decodable for the same next token, yet standard decoding discards earlier states. Because earlier loops embody less computation, recurrence inherently supplies aligned weak-and-strong prediction pairs without auxiliary models or external training. We introduce LoopCD, a training-free contrastive decoding framework that guides token selection by contrasting the final prediction with an earlier recurrent pass, operating either in logit space with one extra output pass (LoopCD-Logits) or in hidden-state space with zero output overhead (LoopCD-Hidden). Across four looped Transformer families, LoopCD delivers substantial, consistent gains at full recurrent depth: LoopCD-Logits raises Ouro-2.6B-Thinking's AIME 2024 pass@1 from 61.88% to 73.33%, while LoopCD-Hidden lifts Huginn's HumanEval pass@1 from 22.56% to 31.71%. Crucially, these performance gains enable halving the number of recurrent loops while still matching or exceeding full-depth unguided baselines, reducing forward FLOPs by 22.5% to 48.2%. By transforming intermediate recurrent states into effective guidance signals, LoopCD achieves superior decoding quality while substantially reducing inference compute.\n    \n\n### References & Citations\n\nLoading...\n\n# Bibliographic and Citation Tools\n\nBibliographic Explorer \n\n*(*[What is the Explorer?](https://info.arxiv.org/labs/showcase.html#arxiv-bibliographic-explorer))\nConnected Papers \n\n*(*[What is Connected Papers?](https://www.connectedpapers.com/about))\nLitmaps \n\n*(*[What is Litmaps?](https://www.litmaps.co/))\nscite Smart Citations \n\n*(*[What are Smart Citations?](https://www.scite.ai/))\n# Code, Data and Media Associated with this Article\n\nalphaXiv \n\n*(*[What is alphaXiv?](https://alphaxiv.org/))\nCatalyzeX Code Finder for Papers \n\n*(*[What is CatalyzeX?](https://www.catalyzex.com))\nDagsHub \n\n*(*[What is DagsHub?](https://dagshub.com/))\nGotit.pub \n\n*(*[What is GotitPub?](http://gotit.pub/faq))\nHugging Face \n\n*(*[What is Huggingface?](https://huggingface.co/huggingface))\nScienceCast \n\n*(*[What is ScienceCast?](https://sciencecast.org/welcome))\n# Demos\n\n# Recommenders and Search Tools\n\nInfluence Flower \n\n*(*[What are Influence Flowers?](https://influencemap.cmlab.dev/))\nCORE Recommender \n\n*(*[What is CORE?](https://core.ac.uk/services/recommender))\nIArxiv Recommender\n\n*(*[What is IArxiv?](https://iarxiv.org/about))\n# arXivLabs: experimental projects with community collaborators\n\narXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.\n\nBoth individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.\n\nHave an idea for a project that will add value for arXiv's community? [**Learn more about arXivLabs**](https://info.arxiv.org/labs/index.html).", "url": "https://wpnews.pro/news/decoding-looped-transformers-better-for-almost-free", "canonical_source": "https://arxiv.org/abs/2610.02185", "published_at": "2026-10-04 02:48:59+00:00", "updated_at": "2026-10-04 03:06:36.341037+00:00", "lang": "en", "topics": ["machine-learning", "large-language-models", "ai-research", "artificial-intelligence"], "entities": ["arXiv", "LoopCD", "Ouro-2.6B-Thinking", "Huginn", "AIME 2024", "HumanEval"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/decoding-looped-transformers-better-for-almost-free", "markdown": "https://wpnews.pro/news/decoding-looped-transformers-better-for-almost-free.md", "text": "https://wpnews.pro/news/decoding-looped-transformers-better-for-almost-free.txt", "jsonld": "https://wpnews.pro/news/decoding-looped-transformers-better-for-almost-free.jsonld"}}