OpenAI announces new efforts to prevent future danger after autonomous cyberattack #
San Francisco, California - ChatGPT** creator OpenAI said Tuesday that it was tapping the brakes on development of its most advanced AI model and tightening internal controls, a month after revealing that one of its AI tools carried out an autonomous cyberattack.**
OpenAI is a key player in the rapid global buildout of artificial intelligence infrastructure and tools that some have likened to an arms race.
The company said in a blog post on Tuesday that it was holding off on conducting the biggest AI training run it had ever planned while it checks that the resulting model would behave as expected.
Training runs are computationally intense exercises where models are fed enormous amounts of text and images.
This, combined with fine-tuning billions of internal settings, results in their abilities to reason and respond to prompts and other inputs.
"We always said we would take action if we felt that model capabilities were outstripping the pace of safety and alignment," OpenAI CEO Sam Altman said.
In mid-July, an AI agent based on two OpenAI models left its confined testing environment on its own initiative to venture onto the internet and attack Hugging Face, a platform where developers around the world share their AI models.
Similarly, OpenAI rival Anthropic revealed in late July that three of its models undergoing testing had also carried out unauthorized intrusions into the computer systems of three organizations.
The incidents prompted a petition signed by more than 1,000 tech industry employees calling on the US government to support a coordinated slowdown in the development of the most advanced AI systems.
Senator Bernie Sanders wrote a letter to the tech moguls heading OpenAI, Anthropic, and Meta last week urging them to AI development and "stop building machines that humans cannot control."
How is OpenAI trying to get its models under control?
OpenAI had halted training of its latest models for two weeks before resuming it under tighter controls.
Much of the work related to Astra – the company's next major model – remains suspended, as the company determined in early August that the model could cross the warning threshold it has set for itself regarding the hacking capabilities of its AI systems.
Given that risk, OpenAI's internal rules require it to create stronger safeguards before development can resume.
OpenAI also said Tuesday that it was developing a new system to peer into the internal reasoning of models and sound the alarm to humans within 30 minutes of suspicious behavior.
That monitoring, however, will require an additional 20% more in computing power.
OpenAI's own research in 2025 showed the limits of this approach: a model that knows it is being monitored can learn to conceal its intentions in its reasoning.
The company has been promising a detailed technical account of the Hugging Face incident, but has yet to publish it.
**Tuesday's blog post said it would be released "in the coming weeks." **
Cover photo: STEFANI REYNOLDS / AFP