Workers at some of the biggest companies in AI are calling for their employers to slow the technology's development.
On Tuesday, a coalition of more than 1,100 employees at OpenAI, Anthropic, Meta, Google and more unveiled a campaign called Pacing the Frontier, urging that the government support the development of the technical and governance tools needed to "deliberately pace the frontier of automated AI development."
The letter said that, as AI labs believe they are nearing the ability to automate AI research, the industry runs the risk of AI's capabilities growing out of our control and beyond our ability to understand it. And because every country and company is under fierce competitive pressure to stay ahead, slowing down must be an industry-wide effort.
"The world is locked in a deadly race towards an intelligence explosion, where AI’s ability to create better AIs reaches a critical point, just like a runaway nuclear chain reaction," Leo Gao, a member of technical staff at OpenAI, wrote in the letter. "Going slower would give us much-needed time to make it go well … to survive, we must coordinate to slow down the race."
The petition is the latest signal that these companies are worried about the consequences of the technology they're creating:
The Information reported on Tuesdaythat leaders at rivals Anthropic and OpenAI are teaming up to ensure that competitors, such as xAI and Meta, are required to comply with government reviews of their frontier models in the same way Mythos and GPT-5.6 were recently.- Additionally, Google DeepMind CEO Demis Hassabishas proposed a new international watchdog organization to perform rigorous safety tests on frontier AI models prior to their release. - And Anthropic called for a large-scale on the development of AI in a recent research blog post on recursive self-improvement, or self-building AI. However, the company argued that one lab pausing its research would not accomplish much, as it would simply change who the frontrunner is, but "not create the wider deliberative process that is currently missing."
These warning sounds come as the formidable capabilities of frontier models come sharply into focus. For instance, OpenAI's accidental breach of Hugging Face marked the first time a powerful model ever escaped from the grasp of its creators. And according to Anthropic, Mythos is capable of finding weaknesses in some of the toughest cryptographic algorithms, the cybersecurity methods used to keep data private online.
Our Deeper View #
Even without AI being able to improve itself, the current capabilities of these models are alarming if you consider what can be done if they get into the wrong hands. It's why companies like Microsoft and Cisco are racing to come up with affordable solutions to help enterprises bolster their cyber defenses. The truth of the matter is that enterprises, organizations and nations are barely ready to grapple with the AI that we currently have. So if models can iterate on themselves rapidly, even the best minds in the industry are unlikely to keep up. Still, while it's commendable that these employees are taking a stand in the face of something potentially dangerous, the reality of an industry-wide feels unlikely. Even though a slowdown is possible in theory, especially when you consider that much of China's success in AI may be piggybacking off of US models, getting every player to agree on a coordinated effort would be a difficult sell considering the massive financial incentives they have to take the lead in this race.