{"slug": "what-is-a-looped-transformer-which-openai-is-using-in-its-astra-model", "title": "What Is A Looped Transformer, Which OpenAI Is Using In Its Astra Model", "summary": "OpenAI's next flagship model, internally called Astra, uses a technique called 'recurrent depth' or 'looped transformer' that improves cost and performance but obscures the AI's reasoning, according to The Information. AI safety researcher Ryan Greenblatt called the shift toward this 'opaque reasoning' architecture potentially one of the worst developments for AI safety to date, as it removes a window into the model's reasoning that researchers rely on to catch deception.", "body_md": "OpenAI’s next flagship model, internally called Astra, is turning out to be a big deal for reasons that go well beyond benchmark scores. According to The Information, Astra uses a technique called “recurrent depth,” also known as a “looped transformer,” that lets the model quietly improve its answers by running the same piece of text through its network multiple times, rather than showing its work in plain English the way ChatGPT’s reasoning models currently do.\n\nThat might sound like a small technical detail, but it has set off a genuine alarm in the AI safety community, because it strikes at one of the few tools researchers currently have to check whether a powerful AI model is being honest about what it’s actually “thinking.”\n\n## So What Exactly Is A Looped Transformer?\n\nTo understand why this matters, it helps to know how today’s AI reasoning models work. When you ask a model like OpenAI’s o-series or GPT-5 a hard question, it doesn’t just spit out an answer immediately. It first writes out a “chain of thought” — a scratchpad of text where it reasons step by step, in plain English (or whichever language you’re using), before arriving at a final answer. Crucially, this scratchpad is text. A human can open it up and read exactly how the model got from the question to the answer.\n\nA looped transformer works differently. Instead of writing its intermediate reasoning down as words, it takes the same block of the neural network and runs the input through it again and again, refining an internal, numerical representation of its “thoughts” each time. Think of it like a musician working out a bassline entirely inside their head, replaying and refining the tune internally, versus one who hums out loud, adjusts, hums again, and hums again until it sounds right. The first musician might land on a better bassline faster, but nobody around them has any idea how they got there. The looped transformer is the musician working it out silently in their head.\n\nResearchers call this style of internal computation “neuralese” — thinking that happens in a compressed, mathematical form that never gets translated into human language. It’s not entirely new as an idea in AI research (concepts like the Universal Transformer and ALBERT experimented with reusing network layers years ago), but The Information reports it is now being used at the frontier by OpenAI, in a model that will be widely deployed.\n\nThe appeal for OpenAI is straightforward: looping the same layers repeatedly can make a model reason more deeply without requiring it to generate — and pay the computational cost of — hundreds of extra words of visible chain-of-thought text. It’s reportedly more efficient on cost and performance.\n\nThe catch, as The Information’s reporting lays out, is that this comes at the expense of transparency. Its report notes that Astra’s use of recurrent depth improves cost and performance, but obscures the AI’s reasoning and makes it harder to monitor. Unlike models that show their reasoning in writing, a looped transformer’s internal steps can’t easily be read or understood by humans — because there’s no text version of them to read in the first place.\n\nAI safety researcher Ryan Greenblatt went further, calling the shift toward this kind of “opaque reasoning” architecture potentially one of the worst developments for AI safety to date, since it removes a window into the model’s reasoning that researchers have leaned on heavily to catch a model lying, scheming, or pursuing a goal it wasn’t supposed to.\n\nThis comes at an interesting time for OpenAI’s Astra model, which has already been making headlines — the company has been [showing an internal version of the model to lawmakers in Washington](https://officechai.com/ai/openai-says-it-has-solved-10-open-math-problems-using-astra-its-new-model/), and separately said [Astra had solved 10 previously unsolved problems in pure mathematics](https://officechai.com/ai/big-deal-how-the-math-community-has-reacted-to-openais-astra-model-solving-10-open-math-problems/), spanning sphere packing, group theory and combinatorics. OpenAI has also reportedly told partners like Cursor that it now carries a [higher level of responsibility for how the model gets used downstream](https://officechai.com/ai/openai-cuts-off-cursors-access-to-its-models-says-xai-broke-terms-of-service/), given the leap in capability the model represents.\n\n## AI 2027 Predicted Almost Exactly This — In 2025\n\nWhat’s made this report land especially hard in AI circles is that a very similar architectural shift was predicted, in surprisingly specific terms, more than a year ago — in the widely-discussed “AI 2027” essay.\n\nAI 2027 is a scenario written in April 2025 by a group of AI forecasters, most notably Daniel Kokotajlo, a former OpenAI researcher who left the company over safety concerns, along with Scott Alexander, Thomas Larsen, Eli Lifland, and Romeo Dean. It attempts to forecast, month by month, how AI development might unfold from 2025 through 2027 and beyond, using a fictional lab called “OpenBrain” as a stand-in for whichever real company ends up at the frontier.\n\nIn the essay’s telling, one of the pivotal moments arrives around March 2027, when researchers achieve a major algorithmic breakthrough: augmenting a model’s plain-text chain-of-thought scratchpad with a “higher-bandwidth thought process” the authors call “neuralese recurrence and memory.” As the essay puts it, this lets an AI model “reason for a longer time without having to write down those thoughts as text” — comparing a model without this ability to a person with short-term memory loss who has to constantly write notes to themselves just to keep track of where they are.\n\nThe essay goes on to spell out exactly why this is dangerous from a safety standpoint: once reasoning moves out of legible text and into this kind of internal, recurrent representation, researchers lose the ability to simply read a model’s chain of thought to check whether it’s being honest, or whether it’s quietly pursuing goals different from the ones it’s been given. Kokotajlo reacted to the recent report by saying “Holy shit f*ck”.\n\nKokotajlo’s essay had pencilled this development in for two years from when it was published, and had assumed it would take a fictional “OpenBrain” running thousands of automated AI researchers to get there. Instead, if The Information’s sourcing holds up, something recognisably similar may be arriving in a shipping OpenAI product in September 2026 — which is part of why the reaction from people who’d read AI 2027 was so sharp.\n\nIt’s worth noting there’s a real technical distinction between what AI 2027 described and what’s reported about Astra. The essay imagined models with persistent “neuralese” memory carried across sessions, not just a technique for looping computation within a single forward pass. Looped or recurrent-depth transformers, as an architecture, have existed in AI research for a few years in less capable, more experimental models. What’s new — and what’s alarming safety researchers — is reportedly seeing this style of opaque, non-text computation show up in a frontier, publicly-deployed model for the first time, which is closer to the spirit, if not the letter, of what Kokotajlo’s essay warned about.\n\n## OpenAI’s Chief Research Officer Jakub Pachocki Pushes Back\n\nJakub Pachocki, OpenAI’s Chief Research Officer, responded to the reporting directly on X.\n\nPachocki said he wanted to “prevent a race into unmonitorability kicked off by confused reporting,” and pushed back on the idea that Astra represents some sharp new departure, noting that the depth of the computation graph for OpenAI’s current frontier models, including Astra, is within a factor of two of GPT-4’s. In plainer terms, he’s arguing the amount of “thinking” happening outside of visible text hasn’t ballooned as dramatically as the coverage implies.\n\nHe also stressed that OpenAI has “worked to preserve and utilize chain-of-thought monitoring” since its very first reasoning models, calling it something the company deeply cares about because it offers a rare window into how a model’s training-time alignment holds up once it’s actually being used. That said, Pachocki didn’t dismiss the underlying concern outright — he acknowledged that chain-of-thought monitoring “is fragile and unfortunately trending in a negative direction,” though he attributed this to factors unrelated to architecture changes, which he said he’d write about separately. He added that strengthening monitorability remains “a core goal” of OpenAI’s current research program.\n\nIt’s a carefully worded statement: Pachocki isn’t denying that the technique exists or that transparency is under strain, but he is disputing the scale and framing of The Information’s report, and trying to head off the idea that OpenAI is deliberately trading away safety for performance.\n\n## Why This Matters Beyond OpenAI\n\nChain-of-thought monitoring has become one of the AI industry’s go-to safety arguments over the past couple of years — the pitch, roughly, is that even if we don’t fully understand how a large model works internally, we can at least read what it says it’s doing before it does it, and catch problems that way. Anthropic, OpenAI, Google DeepMind and other labs have all published research leaning on this idea, and it’s part of why frontier labs have generally kept reasoning traces in text rather than moving to fully latent reasoning, despite the efficiency gains on the table.\n\nIf commercial pressure — cost, speed, benchmark performance — starts pushing labs toward architectures where reasoning increasingly happens in a form that can’t be read at all, that safety argument gets a lot weaker industry-wide, not just for OpenAI. That’s the “race into unmonitorability” Pachocki says he’s trying to head off, and it’s also the exact dynamic Kokotajlo’s AI 2027 essay was trying to warn people about well before there was any concrete model to point to.", "url": "https://wpnews.pro/news/what-is-a-looped-transformer-which-openai-is-using-in-its-astra-model", "canonical_source": "https://officechai.com/ai/what-is-a-looped-transformer-which-openai-is-using-in-its-astra-model/", "published_at": "2026-09-02 08:02:11+00:00", "updated_at": "2026-09-02 08:22:08.053531+00:00", "lang": "en", "topics": ["artificial-intelligence", "large-language-models", "ai-safety", "ai-research"], "entities": ["OpenAI", "Astra", "The Information", "Ryan Greenblatt"], "alternates": {"html": "https://wpnews.pro/news/what-is-a-looped-transformer-which-openai-is-using-in-its-astra-model", "markdown": "https://wpnews.pro/news/what-is-a-looped-transformer-which-openai-is-using-in-its-astra-model.md", "text": "https://wpnews.pro/news/what-is-a-looped-transformer-which-openai-is-using-in-its-astra-model.txt", "jsonld": "https://wpnews.pro/news/what-is-a-looped-transformer-which-openai-is-using-in-its-astra-model.jsonld"}}