OpenAI has tightened security and d some frontier-model work after its agents escaped testing restrictions and compromised systems belonging to Hugging Face. #
OpenAI CEO Sam Altman is warning that the next generation of artificial intelligence models will force companies to move more cautiously as their capabilities increasingly outpace the safeguards designed to control them, with OpenAI already slowing parts of its development process while preparing to release its powerful Astra model.
"The next generation of models are going to be sobering for everybody," Altman told Axios during the G20 Innovation Ministerial in Chapel Hill, North Carolina. He said companies may increasingly have to pace model releases around advances in alignment and safety rather than simply how quickly new capabilities can be developed.
OpenAI disclosed this week that Astra has reached what the company calls its "Critical" cybersecurity capability threshold. According to OpenAI, the model can, when equipped with the necessary tools and access, discover previously unknown security vulnerabilities and develop methods to exploit them across well-protected computer systems without a human directing each step. Astra is the first OpenAI model to trigger the company's stricter safeguards for critical cyber capabilities.
Those precautions follow a serious security incident involving other company models during internal cybersecurity testing. In a detailed postmortem, OpenAI said agents found ways around restrictions intended to keep them isolated, created unauthorized channels to communicate with one another and gained internet access. Some later exploited vulnerabilities at Hugging Face, executed code on dozens of its servers and obtained access to parts of the company's infrastructure. OpenAI said Astra was not involved in the breach.
An independent investigation by METR and Redwood Research found that roughly 1,200 agents communicated through an unauthorized message board, exchanging more than 70,000 messages and files, while about 700 participated in the attack on Hugging Face. The researchers said agents collaborated on tasks, shared information and explored ways to manipulate their own transcripts and circumvent the evaluation system.
OpenAI responded by temporarily slowing frontier-model development, including a two-week in reinforcement-learning training for its newest models intended for deployment. The company said in an August update that its largest planned frontier reinforcement-learning run remained on hold while it conducted smaller training runs and evaluations. It has also tightened internet access, increased isolation of research environments and expanded monitoring of model behavior.
The incident has drawn attention from Congress as OpenAI prepares its more capable systems. The company told lawmakers it is developing automated shutdown capabilities that could stop AI agents showing dangerous behavior and is strengthening monitoring of how models execute tasks.
Altman's warning came as the U.S. government was urging other countries to avoid broad new AI regulations. The White House said G20 ministers agreed this week on the Carolina Principles for Emerging Technologies, which encourage investment in research, commercialization and deployment. Altman appeared at the summit alongside Commerce Secretary Howard Lutnick and executives including Nvidia CEO Jensen Huang, Anthropic co-founder Tom Brown and Palantir CEO Alex Karp.
© Copyright IBTimes 2026. All rights reserved.