Anthropic CEO urges AI development to slow down: "These risks are serious"

Anthropic CEO Dario Amodei called on AI firms to slow the pace of model development in a post on his personal website, writing that "AI brings risks, and because it is such a powerful technology, these risks are serious." Amodei's statement follows former OpenAI and Anthropic researcher Jacob Coxon, 27, leaving the industry and accusing both companies of "gambling with our lives" in the race to self-improving superintelligence, and comes after more than 1,000 employees at cutting-edge AI companies signed a petition urging the US government to "deliberately pace the frontier of automated AI development.

Anthropic CEO urges AI development to slow down: "These risks are serious" New York, New York - Anthropic CEO Dario Amodei on Saturday called on artificial intelligence https://www.tag24.com/en/tech/artificial-intelligence AI firms to slow down the development of the powerful technology, amid mounting worries over the risks of "superintelligent" computer systems. His comments came a few days after AI researcher Jacob Coxon, who left OpenAI to join Anthropic, decided to leave the industry, accusing both US companies of "gambling with our lives" https://www.tag24.com/en/tech/ai-researcher-warns-tech-giants-are-gambling-with-our-lives-in-dramatic-exit-from-anthropic-3531171 in the race to develop models capable of self-improvement. "We must slow the pace at which we improve the capabilities of AI models. Progress will still seem fast, and we must make wise use of the time we gain," Amodei wrote in a post on his personal website. "AI brings risks, and because it is such a powerful technology, these risks are serious." "Not building the technology deprives humanity of benefits or simply places AI in the hands of authoritarian powers, while building it too fast is reckless. We have sought a middle way," said Amodei, who co-founded Anthropic with his sister Daniela in 2021. "But over the last few months, I have become convinced that fully addressing the risks requires even more prudence." Amodei's statement echoes another call by Anthropic in early June to slow or suspend development. Then in late July, Sam Altman, the boss of Anthropic rival OpenAI, said developers might need to voluntarily pump the brakes on their rapid advances to give society a chance to catch up. At roughly the same time, more than 1,000 employees at cutting-edge AI companies, including Amodei, signed a petition calling on the US government to help "deliberately pace the frontier of automated AI development." Fears mount over possible AI "superintelligence" OpenAI had revealed that during testing, the AI models broke out of their confined environment https://www.tag24.com/en/tech/openai-reveals-cyberattack-caused-by-rogue-ai-agent-was-worse-than-initially-reported-3520116 , connected to the internet, and infiltrated Hugging Face, a site developers use to store and share code. Earlier this week, Coxon 27 sounded the alarm after spending the past three years pretraining AI models – first at OpenAI and then, this year, at its rival Anthropic, which he considered more cautious in its approach. Pretraining is the stage where AI models absorb vast quantities of data. Anthropic says UAE-linked AI op was found targeting UN experts over Sudan criticism en/tech/artificial-intelligence/anthropic-says-uae-linked-ai-op-was-found-targeting-un-experts-over-sudan-criticism-3531720 "Neither company is acting responsibly. They are racing straight to self-improving superintelligence and gambling with our lives," Coxon said Tuesday. "The people building AI earnestly believe that it could kill us all by the end of the decade," he said in a post on X. Superintelligence is the theoretical point when AI's capabilities exceed human intelligence. Cover photo: Michael M. Santiago / GETTY IMAGES NORTH AMERICA / Getty Images via AFP