Microsoft CEO Satya Nadella weighed in on the increasingly contentious AI safety debate this week, urging companies to “do what it takes to build stuff that serves humanity first and is in human control” and calling for thorough testing, including by outside testers.
Nadella made the comments Tuesday at the All-In Summit in Los Angeles, a day after he warned developers against rushing AI products to market.
“As the stakes get higher, one should take all the time they need,” he wrote. “If not, you will anyway lose permission to operate.”
Several top tech executives have recently commented on AI safety amid growing concerns about the pace of development.
After researcher Jacob Coxon resigned from Anthropic last week and accused leading AI labs of “gambling with our lives,” Anthropic CEO Dario Amodei published an essay calling for an industry-wide slowdown in frontier AI development. OpenAI’s Sam Altman and other top tech executives backed the call.
Nvidia CEO Jensen Huang and Meta CEO Mark Zuckerberg, however, have expressed more market-driven views, arguing that AI labs are already incentivized to police themselves due to liability concerns.
“If you build a product or a service and you’re not confident in its functionality, capability or safety, then don’t release it,” Huang said yesterday.
Nadella made a similar point yesterday, arguing that engineers can mitigate many AI risks using common sense and established quality control processes.
“One of the biggest things you learn as an early sort of engineering lead is how to deal with a showstopper bug,” he said. “If you see a showstopper, stop the show.”
He admitted, however, that AI also presents some novel risks: for example, businesses should have transparency into how AI uses their data.
“I want my privacy. I want to be able to embed my knowledge in a set of weights I control,” he said. “My IP shouldn’t leak.”
Nadella also described autonomous AI agents as a new form of “insider risk,” envisioning a scenario where an AI agent decides to “fake my books” to complete a task it has been instructed to do. He added that long-running agents require not only containment, but “true aggressive monitoring of agent activity.”
In the long run, Nadella envisioned the possibility of international AI safety norms, particularly if the risks associated with AI can be clearly defined. If something really goes wrong, he said, “it’s going to go wrong everywhere at the same time.”