The world’s biggest tech leaders are drawing lines in the sand over whether AI development should slow its progress to protect against a potential catastrophe. Mark Zuckerberg just chose his side, distancing himself from the camp of AI safety hawks (and chief among them Anthropic CEO Dario Amodei) who have called for AI labs to pace their development to mitigate the risk of an advanced, superintelligent AI spiraling out of human control.
In a social media post Tuesday, the Meta CEO implied slowing down AI development isn’t necessary because frontier AI companies have other incentives, including market forces, that will keep them in check. And ultimately, that comes down to the customer being always right.
“People won’t want to use agents that are misaligned with them and that don’t do what they ask, so labs have a strong natural incentive to make their models more aligned,” Zuckerberg wrote.
The CEO added that his own company delayed the release of Muse, the AI agent it launched earlier this month, because of safety and security concerns. That decision, he said, showed how some companies are already self-policing when it comes to AI safety.
“We didn’t call for everyone else to do this before we would. We just did it as part of our day-to-day work because it was clearly the right thing for people and for us,” he said in the post.
With his Tuesday comments, Zuckerberg waded into a debate that is deeply dividing tech leaders and executives but has taken on new importance given recent security incidents. This summer, a pair of hacks executed by OpenAI models that escaped training environments raised alarm bells over what safety protocols are in place for AI agents. Then last week, momentum built for regulation following a warning by former Anthropic researcher Jacob Coxon that AI companies are “gambling with our lives” when it comes to their development of superintelligent AI.
Despite recent developments, Zuckerberg has now aligned himself closer to the sentiments shared by Nvidia CEO Jensen Huang, who has been among the loudest voices rejecting calls from Amodei and others to slow AI development.
AI safety clashes #
The two differences in opinion were made clear when both Amodei and Huang spoke at Salesforce’s Dreamforce event Tuesday.
Amodei at the conference reiterated his calls for a slowdown on AI development by comparing the safety situation AI companies are facing to those faced by the automotive industry.
He argued that when one company faces a safety incident, other companies should examine their own operations rather than blame others.
“It’s very tempting to attack your competitor and say these guys are unsafe,” he said. “But I think the more responsible way to respond to it is to say, let’s look at our own record. We may not have had this big, high-profile incident, but I’m sure we’re not perfect.”
While Amodei has previously said he is not in favor of an all-out halt in model training or progress, he noted that even with a theoretical freeze on AI, companies are still “making use of maybe only 5% or 10% of what the possible value of the technology is.” Huang took the stage later and made the opposite case. He said safety should remain a top priority but argued that companies can solve those problems themselves. New laws or some sort of industry-wide slowdown just aren’t necessary, he argued.
If a company notices a product isn’t safe, they should and take steps to make sure it is, but otherwise, companies should “run as fast as they can,” he said. The Nvidia CEO added that he didn’t believe companies needed to choose between moving quickly or creating safe products. “Innovation, speed, and safe products—it’s a false choice,” he said. “You could definitely have both at the same time.”
Zuckerberg seems to agree. He said Meta is committing a significant majority of its computing power toward serving people rather than racing to develop AI systems capable of recursively improving themselves.
“I believe the key to building a positive future for everyone is maintaining the right balance of power. This is within our power to do.”
breaks the traditional barrier between audience and newsroom. The show transforms
Fortune Daily Fortune’s trusted reporting into actionable, conversational, and entertaining insights for an emerging class of business leaders.
Watch here.