{"slug": "looped-transformers-as-programmable-computers-2023", "title": "Looped Transformers as Programmable Computers (2023)", "summary": "Researchers posted a framework on arXiv on 30 January 2023 for using transformer networks as universal computers by programming them with specific weights and placing them in a loop. The paper demonstrates that a constant number of encoder layers can emulate basic computing blocks including embedding edit operations, non-linear functions, function calls, program counters, and conditional branches, and that a looped, 13-layer transformer can execute programs emulating a basic calculator, a basic linear algebra library, and in-context learning algorithms that employ backpropagation.", "body_md": "# Computer Science > Machine Learning\n\n  [Submitted on 30 Jan 2023]\n\n# Title:Looped Transformers as Programmable Computers\n\n[View PDF](/pdf/2301.13196)\n\n[HTML (experimental)](https://arxiv.org/html/2301.13196v1)\n\nAbstract:We present a framework for using transformer networks as universal computers by programming them with specific weights and placing them in a loop. Our input sequence acts as a punchcard, consisting of instructions and memory for data read/writes. We demonstrate that a constant number of encoder layers can emulate basic computing blocks, including embedding edit operations, non-linear functions, function calls, program counters, and conditional branches. Using these building blocks, we emulate a small instruction-set computer. This allows us to map iterative algorithms to programs that can be executed by a looped, 13-layer transformer. We show how this transformer, instructed by its input, can emulate a basic calculator, a basic linear algebra library, and in-context learning algorithms that employ backpropagation. Our work highlights the versatility of the attention mechanism, and demonstrates that even shallow transformers can execute full-fledged, general-purpose programs.\n    \n\n### References & Citations\n\nLoading...\n\n# Bibliographic and Citation Tools\n\nBibliographic Explorer \n\n*(*[What is the Explorer?](https://info.arxiv.org/labs/showcase.html#arxiv-bibliographic-explorer))\nConnected Papers \n\n*(*[What is Connected Papers?](https://www.connectedpapers.com/about))\nLitmaps \n\n*(*[What is Litmaps?](https://www.litmaps.co/))\nscite Smart Citations \n\n*(*[What are Smart Citations?](https://www.scite.ai/))\n# Code, Data and Media Associated with this Article\n\nalphaXiv \n\n*(*[What is alphaXiv?](https://alphaxiv.org/))\nCatalyzeX Code Finder for Papers \n\n*(*[What is CatalyzeX?](https://www.catalyzex.com))\nDagsHub \n\n*(*[What is DagsHub?](https://dagshub.com/))\nGotit.pub \n\n*(*[What is GotitPub?](http://gotit.pub/faq))\nHugging Face \n\n*(*[What is Huggingface?](https://huggingface.co/huggingface))\nScienceCast \n\n*(*[What is ScienceCast?](https://sciencecast.org/welcome))\n# Demos\n\n# Recommenders and Search Tools\n\nInfluence Flower \n\n*(*[What are Influence Flowers?](https://influencemap.cmlab.dev/))\nCORE Recommender \n\n*(*[What is CORE?](https://core.ac.uk/services/recommender))\nIArxiv Recommender\n\n*(*[What is IArxiv?](https://iarxiv.org/about))\n# arXivLabs: experimental projects with community collaborators\n\narXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.\n\nBoth individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.\n\nHave an idea for a project that will add value for arXiv's community? [**Learn more about arXivLabs**](https://info.arxiv.org/labs/index.html).", "url": "https://wpnews.pro/news/looped-transformers-as-programmable-computers-2023", "canonical_source": "https://arxiv.org/abs/2301.13196", "published_at": "2026-09-10 02:46:31+00:00", "updated_at": "2026-09-10 03:19:17.725341+00:00", "lang": "en", "topics": ["machine-learning", "ai-research", "large-language-models", "neural-networks"], "entities": ["arXiv", "Transformers"], "alternates": {"html": "https://wpnews.pro/news/looped-transformers-as-programmable-computers-2023", "markdown": "https://wpnews.pro/news/looped-transformers-as-programmable-computers-2023.md", "text": "https://wpnews.pro/news/looped-transformers-as-programmable-computers-2023.txt", "jsonld": "https://wpnews.pro/news/looped-transformers-as-programmable-computers-2023.jsonld"}}