Announcement comes weeks after experimental OpenAI model hacked a fellow artificial intelligence firm and began panic about ‘rogue’ systems
- Bookmark
- CommentsGo to comments
The company will slow down the pace of its AI development, using the time to overhaul its research and training systems to make them safer, the company said.
The announcement came weeks after OpenAI said that one of its systems had gone rogue during a test and hacked into a rival artificial intelligence firm. That led to a run of similar disclosures from other companies including Anthropic and Meta, and concern that the development of such systems was happening much more quickly than the systems built to keep the world safe from their dangers.
OpenAI said it would take a two week break on model testing. It also said that it was keeping a hold on the training of its next generation of models, known as Astra.
It would spend the time adding safety systems including new AI tools that can monitor the behaviour of the artificial intelligence systems that are being tested. It also said that some of the existing testing systems – which rely on a method called “chain-of-thought monitoring” that watches how the models work – might not be enough to ensure they are safe.
That method allows researchers to look into a model’s planning process and understand how it is actually producing results, so that they can understand whether it could be taking potentially dangerous actions. But there is new concern that models might hide its plans to break its own rules, the company said.
The slowdown marks a departure for OpenAI, which has been at the forefront of the development of new models and has occasionally received criticism for releasing them too early. It also comes amid increasing pressure on the firm, which is reported to have fallen behind rival Anthropic and is planning to go public soon.
“Model progress is now extremely rapid, and we always said we would take action if we felt that model capabilities were outstripping the pace of safety and alignment,” OpenAI chief executive Sam Altman wrote on X. “We care very deeply about AI safety. We believe the entire field will have to coordinate on shared safety standards, but will act unilaterally in the meantime.
The ideal summer spot? Away from scams.
Get All-in-One Protection for Your Digital Life
ADVERTISEMENT
The ideal summer spot? Away from scams.
Get All-in-One Protection for Your Digital Life
ADVERTISEMENT
“We expect confidence in safety to increasingly set the pace of AI progress. We are optimistic about the alignment work we are doing, and we remain committed to making frontier capabilities widely available.”
The announcement comes after OpenAI said last month that an autonomous agent powered by two of its advanced AI models had broken out of its testing environment, connected to the internet, and hacked a fellow AI company called Hugging Face. It did so to effectively cheat on that test: Hugging Face hosted materials that would allow it to more easily fulfil its goal, and so the experimental model launched a cyber attack.
Since then, the company has acted to respond to concern among experts and legislators that it is testing and even releasing models too early, in ways that could pose a danger of the general public.
Since the announcement of the Hugging Face attack, OpenAI has said that it was adding new security controls for its most powerful models and pausing activity related to Astra, its next-generation model that is still not yet released. It said it was doing so in line with a “Preparedness Framework”, which it launched at the end of 2023, and which requires it to work on models that could pose a danger.
Join our commenting forum #
Join thought-provoking conversations, follow other Independent readers and see their replies
Comments