A brief history of AI executives calling for regulation

AI executives including OpenAI CEO Sam Altman, Anthropic CEO Dario Amodei, Google DeepMind cofounder Demis Hassabis, Microsoft CEO Satya Nadella, and X CEO Elon Musk publicly agreed that AI needs regulation, continuing a pattern of industry calls for legal guardrails that dates back to Elon Musk's July 2017 warning to US governors, when he had already invested $38 million in OpenAI. The article traces earlier warnings from Samuel Butler in 1863, Alan Turing's 1951 lecture, Bill Joy's 2000 essay, and Microsoft researcher Eric Horvitz's 2009 gathering of AI researchers, noting the industry has claimed to want regulation for years with few meaningful results.

Over the past few days, a lot of people who stand to make a lot of money from AI all publicly agreed https://www.theverge.com/ai-artificial-intelligence/995141/ai-executives-politicians-safety-regulation-anthropic-dario-amodei that it’s time to make everyone slow down before we lose control — including OpenAI CEO Sam Altman, Anthropic CEO Dario Amodei, Google DeepMind cofounder Demis Hassabis, Microsoft CEO Satya Nadella, and X CEO Elon Musk. A brief history of AI executives calling for regulation The industry is in a panic, but it’s been claiming to want legal guardrails for years — with few meaningful results. A brief history of AI executives calling for regulation The industry is in a panic, but it’s been claiming to want legal guardrails for years — with few meaningful results. When people who profit from something declare that it’s dangerous and needs to be regulated, there’s always reason to be skeptical https://www.theverge.com/ai-artificial-intelligence/995186/is-big-techs-ai-slowdown-a-safety-pact-or-a-cartel . But whatever their reason, this is far from the first time AI thought leaders have sounded the alarm. Charles Darwin’s theory of evolution prompted author Samuel Butler to warn about intelligent self-replicating machines replacing humans back in 1863 https://en.wikipedia.org/wiki/Darwin among the Machines — and in his follow-up 1872 book Erewhon https://www.gutenberg.org/cache/epub/1906/pg1906-images.html , he describes visiting a fictional society that has already regulated machines out of existence to avoid humanity’s collapse. Computing pioneer Alan Turing famously warned in a 1951 lecture https://turingarchive.kings.cam.ac.uk/publications-lectures-and-talks-amtb/amt-b-4 that artificial intelligence would “meet with great opposition,” because undying machines “would not take long to outstrip our feeble powers” and would eventually “take control.” Calls for intervention, too, aren’t new. In 2000, Sun Microsystems cofounder and chief scientist Bill Joy wrote https://www.wired.com/2000/04/joy-2/ that self-replicating robots could be more dangerous than nuclear weapons, partly because they’re being developed “within the now-unchallenged system of global capitalism” rather than in tightly controlled government labs. And in 2009 https://erichorvitz.com/AAAI Presidential Panel 2008-2009.htm , Microsoft researcher Eric Horvitz, who now serves as the company’s chief scientific officer, brought leading AI researchers together https://www.nytimes.com/2009/07/26/science/26robot.html to consider “creating policies that might constrain or bias the behaviors of autonomous and semi-autonomous systems so as to address concerns.” But modern AI bigwigs warning that their investments should be controlled? That trend appears to have begun with Elon Musk, who invested earlier than most. July 2017 https://www.theverge.com/2017/7/17/15980954/elon-musk-ai-regulation-existential-threat : Elon Musk says we need to regulate AI before it becomes a danger to humanity July 2017 By the time Elon Musk told a gathering of US governors that they needed to regulate AI right away, the SpaceX and Tesla CEO already had skin in the game. He’d already invested $38 million in OpenAI https://www.cbsnews.com/news/elon-musk-sam-altman-openai-trial-oakland/ , as well as stakes in at least two other AI companies, and he’d publicly spoken about the dangers of AI for three years. In June 2014, he suggested AI research could produce a real-life Terminator https://www.theverge.com/2014/6/18/5820880/elon-musk-worried-ai-research-could-produce-real-terminator , called AI “ potentially more dangerous than nukes https://www.theguardian.com/technology/2014/oct/27/elon-musk-artificial-intelligence-ai-biggest-existential-threat ” that August, and suggested that working on AI was “ summoning the demon https://www.theguardian.com/technology/2014/oct/27/elon-musk-artificial-intelligence-ai-biggest-existential-threat ” that October. In January 2015, he joined Stephen Hawking in signing an open letter https://www.theverge.com/2015/1/12/7531269/scientists-sign-open-letter-focusing-on-AI-dangers-benefits asking for responsible AI research; Hawking had previously warned that the creation of AI might be the last thing humans ever do https://www.independent.co.uk/news/science/stephen-hawking-transcendence-looks-at-the-implications-of-artificial-intelligence-but-are-we-taking-ai-seriously-enough-9313474.html . “AI is a rare case where we need to be proactive about regulation instead of reactive. Because I think by the time we are reactive in AI regulation, it’s too late,” Musk told a gathering of US governors at the July 2017 National Governors Association summer meeting. In March 2018 https://www.theverge.com/2018/3/11/17106910/elon-musk-ai-threat-mars-moon-colonization-nukes-sxsw-2018 , he added: “AI is far more dangerous than nukes. So why do we have no regulatory oversight?” March 2018 https://www.theverge.com/2018/3/21/17150270/mark-zuckerberg-facebook-regulated : Mark Zuckerberg is ‘actually not sure we shouldn’t be regulated’ In the wake of Facebook’s Cambridge Analytica data scandal, the Meta founder suggested his company should be regulated — if it’s the “right” regulation. It was largely not about AI, but he did go there: Now that companies increasingly over the next five to 10 years, as AI tools get better and better, will be able to proactively determine what might be offensive content or violate some rules, what therefore is the responsibility and legal responsibility of companies to do that? In March 2019 https://www.theverge.com/2019/3/30/18288470/mark-zuckerberg-op-ed-tech-regulation-privacy-data-portability-harmful-content-elections , he added that the internet needs a “more active role” for regulators. December 2018 https://www.theverge.com/2018/12/7/18129858/microsoft-facial-recognition-ai-now-google : Microsoft sounds an alarm over facial recognition technology Earlier in 2018, Microsoft began taking fire for working with ICE on facial recognition technologies https://www.theverge.com/2018/6/20/17482500/microsoft-ceo-satya-nadella-ice-contract-memo , at a time many tech CEOs had condemned Donald Trump’s policy https://www.theverge.com/2018/6/19/17480072/trump-border-separation-tech-ceos-cook-zuckerberg-airbnb-uber-youtube of separating children from parents who are arrested at the US border. While this form of AI isn’t quite the same as self-replicating intelligence, Microsoft president Brad Smith opened the door to voluntary regulation in a speech at the Brookings Institution, saying, “We believe it’s important for governments in 2019 to start adopting laws to regulate this technology. … We believe that the only way to protect against this race to the bottom is to build a floor of responsibility that supports healthy market competition. And a solid floor requires that we ensure that this technology, and the organizations that develop and use it, are governed by the rule of law.” January 2020 https://www.theverge.com/2020/1/20/21073682/ai-regulation-google-alphabet-ceo-sundar-pichai : Alphabet CEO Sundar Pichai says there is ‘no question’ that AI needs to be regulated January 2020 “ T here is no question in my mind that artificial intelligence needs to be regulated. It is too important not to,” wrote Pichai in an editorial for the Financial Times https://www.ft.com/content/3467659a-386d-11ea-ac3c-f68c10993b04 . “The only question is how to approach it.” “Companies such as ours cannot just build promising new technology and let market forces decide how it will be used,” he added. Like with Microsoft a year earlier, these comments emerged around fears about AI facial recognition specifically. March 2023 https://www.theverge.com/2023/3/29/23661374/elon-musk-ai-researchers-pause-research-open-letter : Elon Musk and top AI researchers call for pause on ‘giant AI experiments’ March 2023 In 2023, as generative AI fervor began to reshape the entire tech industry https://www.theverge.com/23981318/chatgpt-open-ai-launch-anniversary-future , scientists began suggesting that companies should slow down https://www.theverge.com/23664519/ai-industry-pause-open-letter-societal-harms their research — or else. “If such a pause cannot be enacted quickly, governments should step in and institute a moratorium,” the letter https://futureoflife.org/open-letter/pause-giant-ai-experiments/ , published by the nonprofit Future of Life Institute and signed by Musk, among others, reads. Two weeks later, Musk announced he’d created his own new AI company https://www.theverge.com/2023/4/14/23684005/elon-musk-new-ai-company-x . Two years later, he would use DOGE https://www.theverge.com/policy/845838/2025-year-end-government-doge to dismantle chunks of the federal workforce instead of helping to regulate anything. May 16th, 2023 https://www.theverge.com/2023/5/16/23726119/congress-ai-hearing-sam-altman-openai : OpenAI CEO Sam Altman asks Congress to regulate AI May 16th, 2023 Brought before the US Senate for a series of AI hearings, Altman agreed that Congress should create a new agency to regulate artificial intelligence. “We believe that the benefits of the tools we have deployed so far vastly outweigh the risks,” he argued. “However, we think that regulatory intervention by governments will be critical to mitigate the risks of increasingly powerful models.” In another hearing on May 25th https://www.bloomberg.com/news/articles/2023-07-25/anthropic-s-amodei-warns-us-senators-of-ai-powered-bioweapons , Anthropic’s CEO warned US senators that AI could help create bioweapons. May 25th, 2023 https://blogs.microsoft.com/on-the-issues/2023/05/25/how-do-we-best-govern-ai/?utm source=gemini : Microsoft president Brad Smith ‘details five ways governments should consider policies, laws, and regulations around AI’ May 25th, 2023 When OpenAI’s CEO was before Congress, we noted https://www.theverge.com/2023/5/16/23726119/congress-ai-hearing-sam-altman-openai that the tone was friendly and they seemed to be trying to shape legislation together; here is an example of Microsoft trying to shape future regulation as well. One regulatory example: “New laws would require operators of these systems to build safety brakes into high-risk AI systems by design. The government would then ensure that operators test high-risk systems regularly to make certain that the system safety measures are effective.” May 30th, 2023 https://www.theverge.com/2023/5/30/23742005/ai-risk-warning-22-word-statement-google-deepmind-openai : Top AI researchers and CEOs warn against ‘risk of extinction’ in 22-word statement May 30th, 2023 “Mitigating the risk of extinction from AI should be a global priority alongside other societal-scale risks such as pandemics and nuclear war.” While the 22-word statement doesn’t explicitly ask the government to step in, it’s hard to think world leaders wouldn’t raise their eyebrows even higher than before. OpenAI CEO Sam Altman and then-Google DeepMind CEO Demis Hassabis were among the cosigners. September 2023 https://www.theverge.com/2023/9/13/23872478/ai-tech-regulation-leaders-insight-forum : Mark Zuckerberg wants everyone to ‘work together’ on regulation September 2023 In a private Senate meeting, closed to the public and media, tech CEOs reportedly agreed that the AI industry needed regulation. One set of remarks was not so private: Meta CEO Mark Zuckerberg said that “Congress should engage with AI to support innovation and safeguards.” “We think policymakers, academics, civil society, and industry should all work together to minimize the potential risks of this new technology, but also to maximize the potential benefits.” Between July 2023 and May 2024, tech companies signed a flurry of nonbinding agreements with governments that would supposedly keep them in check: - July 2023 https://www.theverge.com/2023/7/21/23802274/artificial-intelligence-meta-google-openai-white-house-security-safety : Meta, Google, and OpenAI promise the White House they’ll develop AI responsibly - September 2023 https://www.theverge.com/2023/9/12/23870092/nvidia-ibm-adobe-white-house-ai-agreement-nonbinding : More companies commit to the White House AI safety accord - November 2023 https://www.reuters.com/world/uk/uk-pm-sunak-lead-ai-summit-talks-before-musk-meeting-2023-11-02/ : At UK’s AI Summit developers and govts agree on testing to help manage risks - May 2024 https://www.euronews.com/next/2024/05/22/ai-seoul-summit-world-leaders-agree-to-launch-network-of-safety-institutes : AI Seoul Summit: World leaders agree to launch network of safety institutes Also in September 2023 https://www.nytimes.com/2023/09/10/business/dealbook/an-ai-leader-urges-regulation-and-a-rethink.html : A book from Inflection AI CEO Mustafa Suleyman — who is now Microsoft’s AI chief — argued that governments should regulate. August–September 2024 https://www.theverge.com/2024/8/23/24226706/anthropics-ceo-has-some-thoughts-about-californias-ai-safety-bill-too : Elon Musk and Anthropic come out in favor of California’s AI safety bill August–September 2024 “This is a tough call and will make some people upset, but, all things considered, I think California should probably pass the SB 1047 AI safety bill,” Musk posted in August https://x.com/elonmusk/status/1828205685386936567?lang=en . Anthropic originally came out against it https://www.axios.com/2024/07/25/exclusive-anthropic-weighs-in-on-california-ai-bill in July, as did OpenAI https://www.theverge.com/2024/8/21/24225648/openai-letter-california-ai-safety-bill-sb-1047 in August, but Anthropic changed its tune in September https://www.theverge.com/2024/8/23/24226706/anthropics-ceo-has-some-thoughts-about-californias-ai-safety-bill-too . California Gov. Gavin Newsom vetoed SB 1047, https://www.theverge.com/2024/9/29/24232172/california-ai-safety-bill-1047-vetoed-gavin-newsom though, stating that “I do not believe this is the best approach to protecting the public from real threats posed by the technology.” October 2024 https://www.anthropic.com/news/the-case-for-targeted-regulation : Anthropic calls for ‘targeted regulation’ October 2024 “ Governments should urgently take action on AI policy in the next eighteen months. The window for proactive risk prevention is closing fast ,” Anthropic wrote in an open letter bolding theirs . “It is critical over the next year that policymakers, the AI industry, safety advocates, civil society, and lawmakers work together to develop an effective regulatory framework,” the company added. January 2025 https://openai.com/global-affairs/openais-economic-blueprint/ : OpenAI submits its ‘policy proposals’ January 2025 The document begins by suggesting the UK stunted the development of its own automobile industry with regulation, while proposing “democratic AI” instead: “Rules and regulations for the development and use of AI should be based on the democratic values the country has always stood for.” June 2025 https://www.nytimes.com/2025/06/05/opinion/anthropic-ceo-regulate-transparency.html : Anthropic CEO: Don’t let AI companies off the hook June 2025 In a guest essay for The New York Times , Anthropic CEO Dario Amodei wrote: “Federal law does not compel us or any other A.I. company to be transparent about our models’ capabilities or to take any meaningful steps toward risk reduction. Some companies can simply choose not to.” The essay is against the Trump administration’s proposed 10-year moratorium https://www.theverge.com/ai-artificial-intelligence/680347/anthropics-ceo-wrote-an-op-ed-against-the-proposed-10-year-moratorium-on-state-ai-regulation on US states regulating AI. He wasn’t arguing for a slowdown, though, but rather a “national transparency standard” that “would help not only the public but also Congress understand how the technology is developing.” September 2025 https://www.anthropic.com/news/anthropic-is-endorsing-sb-53 : Anthropic endorses California’s SB 53 September 2025 As California tried to regulate its AI giants again https://www.theverge.com/ai-artificial-intelligence/688301/california-is-trying-to-regulate-its-ai-giants-again , Anthropic threw its support in early, and this one actually became law https://www.theverge.com/ai-artificial-intelligence/787918/sb-53-the-landmark-ai-transparency-bill-is-now-law-in-california . It’s a transparency standard, like the one Anthropic was asking for, with safety reporting requirements and whistleblower protection. OpenAI lobbied against it. September 2026 https://www.theverge.com/ai-artificial-intelligence/995141/ai-executives-politicians-safety-regulation-anthropic-dario-amodei : The AI industry seemingly agrees to slow down — while Trump calls AI fears ‘a sick conspiracy’ September 2026 This summer, shit hit the fan. It became public that rogue AI agents can https://www.theverge.com/column/980337/rogue-ai-science-fiction-openai — and did https://www.theverge.com/ai-artificial-intelligence/972380/open-ai-hugging-face-hack-ai-safety-warning — attack other companies. OpenAI said it hit the brakes https://www.theverge.com/ai-artificial-intelligence/982323/openai-hit-brakes-voluntary-pacing-ai in August, and on September 8th, a senior Anthropic safety researcher resigned in protest, declaring there’s a 10 percent chance AI could kill all humans by the end of the decade https://www.theverge.com/ai-artificial-intelligence/991927/anthropic-ai-kill-all-humans . Another Anthropic safety researcher agreed. Now, suddenly, the AI industry has seemingly agreed to slow down the pace of development. More: Is this a new call for regulation? From Anthropic’s CEO, it seems to be. In his essay https://darioamodei.com/post/we-must-pace-the-frontier “We Must Pace the Frontier,” Amodei writes that “The most effective method of pacing is via regulation that targets all US frontier AI companies, as that covers even those who are unwilling to cooperate voluntarily,” and he suggests governments might embed people inside AI companies to help prevent incidents. It’s not clear every AI company wants to go that far. “Where we will need the help of our government is for international coordination. But first we should do what we can ourselves,” OpenAI CEO Sam Altman writes https://x.com/sama/status/2099348812305473766 . But Altman does say https://x.com/sama/status/2098811563415150910?lang=en that he’ll commit to “independent evaluators with employee-like access” as well. Meta’s Mark Zuckerberg also doesn’t seem to be aligned. On September 15th, he posted https://x.com/finkd/status/2099997096896274533 that each company has its own individual responsibility to move at the right pace and that Meta is already doing so. He also prominently linked back to his manifesto from August, https://www.theverge.com/tech/977395/meta-mark-zuckerberg-superintelligent-ai-ramble which contains this line: “Any policy that slows American model releases -- even by a month -- could add significant risk to American leadership while letting foreign models race ahead.” Regardless of what tech leaders are saying, regulation seems highly unlikely to come under the Trump administration, which has signaled https://truthsocial.com/@realDonaldTrump/posts/117269745153543631 it wants the industry barreling full speed ahead. Even if these companies don’t actually want the regulation they seem to be asking for, this is one of the loudest alarm bells they’ve sounded so far — and the world is watching. Follow topics and authors from this story to see more like this in your personalized homepage feed and to receive email updates.