Letter signed by almost 1,200 employees of AI companies including OpenAI, Anthropic and Google Deepmind suggests that we might soon be able to ‘understand or control’ powerful tools
- Bookmark
- CommentsGo to comments
Almost 1,200 senior employees at the world’s most powerful AI firms have called for an urgent slowdown in the development of the technology.
The technology is in serious risk of developing so quickly that we are unable to “understand or control the resulting systems”, they say in a new open letter.
The letter, titled ‘Pacing the Frontier’, is signed by leaders and top scientists at some of the world’s most powerful AI companies, including ChatGPT maker OpenAI, Anthropic and Google Deepmind.
It warns that “AI could help create a dramatically better future, but that outcome is not guaranteed”. There is a danger that the technology will develop more quickly than our ability to “address emerging risks, develop security measures, and strengthen oversight”, and mean that the technology spirals out of control, it warns.
As such, the letter’s authors call for an urgent slowdown on work on AI models. But it says that the companies themselves would be unable to do so, and that it will require the US government to support an international effort to “deliberately pace the frontier of automated AI development”.
"If it's true that AI will be as transformational as electricity, we better be intentional about how it is introduced to the world,” said Laura Weidinger, a staff research scientist at Google and one of the signatories of the new letter. “A mindless race without assurances on governance, safety or the technology serving the public interest stands in the way of such intentionality”.
Writing on X in response to the statement, OpenAI agreed that the pace of development of the technology could become a problem. “We believe that, at some point in the future, AI acceleration for frontier model development may be so high that the world will need to pace the rate of AI advancement,” it said.
And its rival Anthropic, which makes the Claude system, said that extra time was needed to prepare. “Our own research on recursive self-improvement, published last month, points to the need for tools to deliberately pace the frontier of AI development so society can prepare," it said in a blog post.
Last month, Anthropic already called on its fellow developers of so-called frontier AI models to slow down or temporarily development if the technology reached that point of “recursive self-improvement”, where systems would be able to improve themselves. It would be “"good for the world to have the option to slow or temporarily frontier AI development to enable societal structures and alignment research to keep up with the advance of the technology," it said in a blog post in June.
OpenAI’s Sam Altman has previously opposed similar petitions, suggesting that they are missing nuance about what parts of development should be d. But he appears to have become more concerned about the threat after an OpenAI model autonomously hacked into another AI company, which he called a ““extremely sci-fi cyber incident” and said it was the “first security incident that I have felt very viscerally”.
“We may have to pace the rate of AI development to give ourselves enough time for society to harden around some of these new capability levels,” he said in an interview with the Invest Like the Best podcast. But he said that must be done “in a way that does not feel like regulatory capture for anyone and also does not feel like collusion among the frontier labs”.
Some critics have however suggested that such claims about the danger of AI development is itself marketing for the companies that create them. Mr Altman has for instance repeatedly used claims about the threat from the technology to raise funding for OpenAI, arguing that empowering companies such as his will help curb the use of AI by bad actors.
‘Pacing the Frontier’ statement in full
“AI could help create a dramatically better future, but that outcome is not guaranteed. The world's leading AI companies believe they could be close to automating AI research. It is hard to predict exactly how much this will accelerate AI progress, but there is a real risk that capability development rapidly accelerates beyond our ability to understand or control the resulting systems.
“To realize AI's potential, industry, government, and society at large may need the option to buy time to address emerging risks, develop security measures, and strengthen oversight. But each company—and country—is under intense competitive pressure not to unilaterally slow that acceleration. And today, the world lacks the technical and governance tools to deliberately pace frontier-wide progress.
“Building on work already underway to monitor frontier model releases: We request that the U.S. government support an international effort to develop the technical and governance tools needed to deliberately pace the frontier of automated AI development.”
Join our commenting forum #
Join thought-provoking conversations, follow other Independent readers and see their replies
Comments