The AI doomsday debate intensifies

Anthropic researcher Jacob Coxon quit last week warning that AI giants are "gambling with our lives," and on Saturday Anthropic CEO Dario Amodei published an essay saying he has "become convinced" the breakneck pace of AI advancement must change, proposing a three-step safety plan that Elon Musk and Sam Altman publicly endorsed. Altman said OpenAI might not move forward with an IPO this year, calling it "an ill-advised moment to go public," as employees at Anthropic and OpenAI debate whether to quit or stay amid AI apocalypse fears.

Existential angst What do you do if you feel like you're helping build something that could contribute to the end of humanity? This is the existential question https://l.businessinsider.com/a/zc/21432/803235dfe02e44df93ec3f836a455086/c2e35828211a477682d9defd773ab364 confronting the AI industry, a debate that only intensified after Jacob Coxon, an Anthropic researcher, quit last week with a stark warning https://l.businessinsider.com/a/zc/21432/803235dfe02e44df93ec3f836a455086/6fa0e79d5e694656b1dea257c448be67 . He said AI giants are "gambling with our lives" and that the people building the technology "earnestly believe that it could kill us all by the end of the decade." The swift reaction to Coxon's warning boiled over on Saturday when the leaders of the biggest AI companies publicly agreed on what should happen next. In a Saturday essay, Anthropic CEO Dario Amodei https://l.businessinsider.com/a/zc/21432/803235dfe02e44df93ec3f836a455086/ae4240cf4d114a808fbc1f89bea93fd4 says he's "become convinced" the breakneck pace of advancement must change and proposed a three-step solution to ensure the tech's safety. Shortly afterward, Elon Musk and Sam Altman publicly aligned with Amodei's plan in a rare sign of support. "Dario is right," Musk tweeted. Altman agreed, too, saying the pace of development has been a "primary topic" of discussion at OpenAI in recent weeks. Altman also said OpenAI might not move forward https://l.businessinsider.com/a/zc/21432/04d69694e8d5427f8e07bb47cb597646/960c29e3bf98410bafd3401eb1facf0d with an IPO this year. "Right now would be an ill-advised moment to go public," he said in a Fortune interview. As fears over AI safety intensify, employees at Anthropic and OpenAI https://l.businessinsider.com/a/zc/21432/04d69694e8d5427f8e07bb47cb597646/031a366d67a44496b37c46e6f41668ba have also been speaking out. My colleague Sarah Needleman https://l.businessinsider.com/a/zc/21432/04d69694e8d5427f8e07bb47cb597646/8dbe59ff83a84f53938fa34889d5fb38 reported on a big question facing many concerned AI employees https://l.businessinsider.com/a/zc/21432/04d69694e8d5427f8e07bb47cb597646/9b26196f86654314ad89faa020594d20 : Should they quit or stay amid fears of an AI apocalypse? Some AI workers believe they can do more good by staying on the inside. That means raising concerns, pushing for safeguards, and trying to influence decisions before they are made. Others worry that staying at these companies only helps legitimize a race that some believe is getting dangerously out of control. BI's Alistair Bar explored some of these theories https://l.businessinsider.com/a/zc/21432/04d69694e8d5427f8e07bb47cb597646/ca5dfed7a4194d4d8fa5540ee252af23 in this week's edition of AI Insider — you can subscribe here https://l.businessinsider.com/a/zc/21432/04d69694e8d5427f8e07bb47cb597646/53c883d8fa0d4f87ac308e730c19db44 to his weekly newsletter. Are employees at the big AI lab companies better off leaving or staying if they feel like their work is contributing to a possible AI apocalypse? Send me a note at srussolillo@businessinsider.com and let me know what you think.