Bridging the Gap Between Latent and Explicit Reasoning with Looped Transformers

Researchers introduced LOTUS, a looped Transformer model that achieves latent chain-of-thought reasoning performance matching explicit CoT at the 3B parameter scale while reducing thought-phase latency by 2.5x to 6.9x. The method uses a recurrent-depth architecture with parallel supervision on latent representations, enabling interpretable and efficient multi-step reasoning without token-by-token generation.

Computer Science Machine Learning Submitted on 30 Jun 2026 v1 https://arxiv.org/abs/2606.31779v1 , last revised 13 Jul 2026 this version, v2 Title:Bridging the Gap Between Latent and Explicit Reasoning with Looped Transformers View PDF /pdf/2606.31779 HTML experimental https://arxiv.org/html/2606.31779v2 Abstract:Language models typically reason via explicit chain-of-thought CoT , generating intermediate steps token-by-token. Latent CoT offers an alternative: it performs multi-step reasoning in the model's hidden states, replacing decoded tokens with continuous representations for greater efficiency. However, existing latent CoT methods underperform explicit CoT beyond 1B parameters, and the gap widens with scale. Looped, or recurrent-depth, Transformers, which reuse their weights to increase computation depth without adding parameters, are a natural fit for latent reasoning. We therefore ask whether looped Transformers can bridge this gap. We answer affirmatively with a simple recipe: a looped padded Transformer that processes K latent blocks in parallel for R iterations, with a cross-entropy loss on each latent position's gold CoT-step token, similar to explicit CoT supervision. We instantiate it as LOTUS Looped Transformers with parallel supervision on latents . LOTUS is, to our knowledge, the first latent-CoT method to bridge the gap to explicit CoT at the 3B scale, while cutting thought-phase latency by 2.5x-6.9x from compact math expressions to natural language. Projecting LOTUS's post-loop latents through the base LM head recovers the gold reasoning steps and even surfaces alternative valid intermediate steps, evidence that its latent space is interpretable and CoT-aligned. Ablations confirm that both the looped backbone and the parallel supervision on gold CoT tokens are essential. Code is available at this https URL . Submission history From: Ying Fan view email /show-email/71381d9f/2606.31779 Tue, 30 Jun 2026 14:58:53 UTC 194 KB v1 /abs/2606.31779v1 v2 Mon, 13 Jul 2026 16:27:19 UTC 187 KB References & Citations Loading... Bibliographic and Citation Tools Bibliographic Explorer What is the Explorer? https://info.arxiv.org/labs/showcase.html arxiv-bibliographic-explorer Connected Papers What is Connected Papers? https://www.connectedpapers.com/about Litmaps What is Litmaps? https://www.litmaps.co/ scite Smart Citations What are Smart Citations? https://www.scite.ai/ Code, Data and Media Associated with this Article alphaXiv What is alphaXiv? https://alphaxiv.org/ CatalyzeX Code Finder for Papers What is CatalyzeX? https://www.catalyzex.com DagsHub What is DagsHub? https://dagshub.com/ Gotit.pub What is GotitPub? http://gotit.pub/faq Hugging Face What is Huggingface? https://huggingface.co/huggingface ScienceCast What is ScienceCast? https://sciencecast.org/welcome Demos Recommenders and Search Tools Influence Flower What are Influence Flowers? https://influencemap.cmlab.dev/ CORE Recommender What is CORE? https://core.ac.uk/services/recommender IArxiv Recommender What is IArxiv? https://iarxiv.org/about arXivLabs: experimental projects with community collaborators arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website. Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them. Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs https://info.arxiv.org/labs/index.html .