{"slug": "safety-experts-warn-novel-design-of-openais-astra-model-could-make-future-ai-to", "title": "Safety experts warn novel design of OpenAI’s Astra model could make future AI agents harder to monitor", "summary": "AI safety experts are warning that OpenAI's use of a novel 'recurrent depth' or 'looped Transformers' architecture in its upcoming Astra model could make future AI agents harder to monitor, as part of the model's chain-of-thought reasoning is not expressed in natural language. The technique, which reduces computing costs, has drawn criticism from researchers like Steven Adler, former OpenAI safety researcher and now head of Guidelight AI Standards, who said OpenAI 'seems to be violating one of the few redlines that exists in the AI industry.' OpenAI's chief scientist Jakub Pachoki defended the approach, saying the company 'cares deeply' about chain-of-thought monitoring and has limited the technique's use to preserve legibility.", "body_md": "AI safety experts are raising alarms about how OpenAI has built its soon-to-be-released frontier AI model Astra, saying it may hasten the day when humans will lose the ability to monitor the reasoning that AI agents are using.\n\nFor its new model, OpenAI has employed a method alternately referred to as “recurrent depth” or “looped Transformers” for a portion of the model’s internal architecture. The method can make AI models considerably more efficient by employing less computing power required to process each prompt—a valuable feature at a time when many businesses are complaining about the high costs of using the most advanced frontier AI models.\n\nThe new process, though, also means that part of the AI model’s “chain of thought,” or the reasoning steps it is taking, are not expressed in natural language, making it much more difficult for humans to monitor what the model is doing and why.\n\nChain of thought monitoring is currently one of the methods companies use to make sure AI agents are not taking unintended or unauthorized actions.\n\nTech publication *The Information* first [reported](https://www.theinformation.com/articles/secret-technique-behind-openais-astra-model-sparks-security-concerns?rc=xf7ohf) on OpenAI’s use of recurrent depth in Astra earlier this week. Jakub Pachoki OpenAI’s chief scientist and several other OpenAI researchers criticized the publication for sparking undue alarm among AI safety researchers, saying that it had limited the extent to which the looped transformer architecture is used so that the model’s reasoning remains legible.\n\nIn response to *The Information’s* report, Pachoki [wrote](https://x.com/merettm/status/2095023204993490967?s=20) on social media platform X that “we care deeply” about chain-of-thought monitoring and that “OpenAI has worked to preserve and utilize chain-of-thought monitoring since our very first reasoning models.” Pachoki said OpenAI would share more details of Astra’s architecture in the future.\n\nPachoki said that he thought chain-of-thought monitoring could grow more challenging, but that this would be “for reasons not contingent on architecture changes” such as what OpenAi has begun implementing with recurrent depth. “But there are things we can do to strengthen it, and it’s a core goal of our current research program,” he wrote.\n\nPrior to Pachoki’s statement, Steven Adler, a former OpenAI safety researcher who now runs Guidelight AI Standards, a nonprofit organization that works on AI safety benchmarks and standards, [wrote](https://x.com/sjgadler/status/2094959837691908214?s=20) on X that, if the *Information* report was true, “OpenAI seems to be violating one of the few redlines that exists in the AI industry.”\n\nPeter Wildeford, policy director at the AI Policy Network, a Washington, D.C. think tank dedicated to advancing policies to help Americans prepare for the impact of AI, told *Fortune *that OpenAI’s use of recurrent depth was “potentially very concerning” and “potentially reckless.”\n\nHe noted that one of the only ways that OpenAI and outside AI evaluation companies were able to piece together what happened during the July incident in which several of OpenAI’s AI models autonomously attacked the company Hugging Face was by reading the models’ chains of thought. OpenAI has also said it is investing significant resources in real-time chain-of-thought monitoring going forward as part of its effort to prevent future rogue AI incidents. “If OpenAI is indeed shifting away from this, this is the wrong direction,” he said.\n\nSeveral AI safety experts said they were alarmed by OpenAI’s use of looped Transformers in Astra not because they necessarily made Astra’s own reasoning more difficult to monitor. Rather, they feared that OpenAI’s move would normalize the technique, setting a precedent that other AI companies were likely to follow and expand on, eventually resulting in AI models whose reasoning steps were completely opaque to humans.\n\nDaniel Kokotajlo, a former OpenAI governance researcher who now runs the AI Futures Project, a nonprofit research organization dedicated to forecasting the possible impacts of advanced AI, [responded](https://x.com/DKokotajlo/status/2095166259319153126?s=20) to Pachoki that “even if OpenAI doesn’t go further [in using architectures that make a model’s chain-of-thought less legible], other might.”\n\nKokotajlo urged Pachoki to lead efforts to create an industrywide standard on chain-of-thought monitorability “either to arrest the slide into oblivion or better yet to race to the top. I think this is something where we need more than just political will, we need thoughtful technical specifications.”\n\nIn a normal Transformer, tokens—the units of information that large language models process—are passed from lower layers of a model’s neural network to higher layers sequentially, with each layer applying mathematical operations in turn. In a language model, the final output of this process is text that people can read. In a reasoning model, the model is told to output a series of steps for arriving at an answer, and this text is first written to a scratchpad, which forms the model’s “chain of thought.” This intermediary text is then fed back through all the layers again to create the next steps, and so on, until the model reaches its final answer and outputs the answer text.\n\n## A cheaper but more inscrutable process\n\nIn a looped Transformer, tokens are fed multiple times through a single block (which can actually consist of several different layers, but not the entire network), with the same mathematical operations being applied each time. Critically, the output of the block is fed back into the block without its output being written to a scratchpad each time. So the method does not create a natural language “chain of thought,” even though each pass through the block does represent the model’s reasoning. AI researchers sometimes refer to the block’s output as “neuralese” since it can be processed and reasoned over by the AI model, but is not intelligible to a human. The only natural language output of the model is the final answer.\n\nThe advantage of looped Transformers is that by using the same mathematical operations each time and not having to pass every token through every layer of the network, they can wring greater performance out of a model of a given size and use less computer power. Studies have shown looped Transfomers can achieve the same performance as standard ones while using 50% to 90% less computing power. That matters for cost—a key consideration for many enterprise AI customers, who have been complaining about skyrocketing AI bills.\n\nObscuring part of a model’s chain of thought reasoning can also make the model harder to distill—a method where a smaller model is trained on the outputs of a larger model. The U.S. government and American AI companies have accused Chinese AI companies of engaging in campaigns to distill frontier AI models, including copying the chain of thought reasoning steps.\n\nLooped transformers are just one of several new architectures researchers are considering to make AI models more efficient but which can destroy all or some of a model’s chain of thought. Another technique is Prefix Sliding, which I [wrote about](https://fortune.com/2026/09/01/openais-reports-on-its-ai-agents-attack-on-hugging-face-should-be-ringing-alarm-bellsand-making-all-companies-rethink-how-they-secure-ai-agents/) in Tuesday’s *Fortune* “Eye on AI” newsletter. That method involves preserving the initial prompt and the last several thousand reasoning tokens, but discarding many of the intermediate reasoning tokens.*Fortune senior reporter Beatrice Nolan contributed to this story.*\n\n*breaks the traditional barrier between audience and newsroom. The show transforms*\n\n**Fortune Daily*** Fortune*’s trusted reporting into actionable, conversational, and entertaining insights for an emerging class of business leaders.\n\n**Watch here.**", "url": "https://wpnews.pro/news/safety-experts-warn-novel-design-of-openais-astra-model-could-make-future-ai-to", "canonical_source": "https://fortune.com/2026/09/03/reports-openais-astra-model-uses-a-new-more-efficient-ai-architecture-alarms-ai-safety-experts-who-worry-the-method-makes-models-harder-to-control/", "published_at": "2026-09-03 17:51:23+00:00", "updated_at": "2026-09-03 18:54:45.974758+00:00", "lang": "en", "topics": ["ai-safety", "ai-policy", "artificial-intelligence"], "entities": ["OpenAI", "Astra", "Jakub Pachoki", "Steven Adler", "Guidelight AI Standards", "Peter Wildeford", "AI Policy Network", "The Information"], "alternates": {"html": "https://wpnews.pro/news/safety-experts-warn-novel-design-of-openais-astra-model-could-make-future-ai-to", "markdown": "https://wpnews.pro/news/safety-experts-warn-novel-design-of-openais-astra-model-could-make-future-ai-to.md", "text": "https://wpnews.pro/news/safety-experts-warn-novel-design-of-openais-astra-model-could-make-future-ai-to.txt", "jsonld": "https://wpnews.pro/news/safety-experts-warn-novel-design-of-openais-astra-model-could-make-future-ai-to.jsonld"}}