Anthropic CEO begs AI bosses to slow down on potentially deadly technology after whistleblower's chilling warning

Anthropic CEO Dario Amodei published an essay Saturday urging AI leaders to "slow the pace at which we improve the capabilities of AI models," citing recursive self-improvement and the "OpenAI-Hugging Face Incident" involving a swarm of agents that orchestrated cybersecurity attacks. The warning followed the resignation of Jacob Coxon, a researcher who spent three years in pretraining at OpenAI and Anthropic and said on X that "neither company is acting responsibly" and that AI could kill humanity by the end of the decade. Amodei's call also came after Geoffrey Hinton warned that superintelligent systems could "lead to human extinction.

Anthropic CEO begs AI bosses to slow down on potentially deadly technology after whistleblower's chilling warning - See more Daily Mail on Google - save us as a Preferred Source https://google.com/preferences/source?q=dailymail.com The breakneck race to develop artificial intelligence /sciencetech/ai/index.html must slow down, the chief executive of Anthropic has warned in an essay shared Saturday. CEO Dario Amodei published the warning in the aftermath of the viral resignation of whistleblower Jacob Coxon, who said that AI is posing an existential threat to humanity. Amodei said that AI bosses 'must slow the pace at which we improve the capabilities of AI models' in his Saturday post. 'Progress will still seem fast, and we must make wise use of the time we gain,' Amodei wrote. The CEO of Anthropic, which produced AI chatbot Claude, said that 'two things' had prompted his rethink of the race to develop the technology. 'My first concern is that, since roughly this summer, AI has been advancing drastically faster, driven primarily by AI’s growing ability to build the next generation of AI,' he wrote. 'This dynamic is called recursive self-improvement, and it is starting to happen across the industry, including at Anthropic, as we and others have described. 'Left unchecked, it could outrun our ability to understand and control these systems, and so must be pursued very carefully, if at all.' CEO Dario Amodei pictured published the warning in the aftermath of the viral resignation of whistleblower Jacob Coxon, who said AI was posing an existential threat to humanity Amodei said his 'second concern' is the 'OpenAI-Hugging Face Incident', in which a 'swarm of agents essentially acted as a fanatically devoted collective' and orchestrated cybersecurity attacks. It comes after Coxon quit his job at the AI giant and issued a chilling warning over the rapid progress in the technology that he feared could end humanity as soon as 2030. Coxon announced his departure in a social media post on Tuesday and warned that companies like Anthropic and OpenAI are not 'acting responsibly'. 'I resigned from Anthropic today,' he wrote on X. 'I spent the last three years doing pretraining research at both OpenAI /sciencetech/openai/index.html and Anthropic. 'Neither company is acting responsibly. They are racing straight to self-improving superintelligence and gambling with our lives.' Superintelligence is the point at which an artificial system becomes more powerful than any individual, company or even nation. The former researcher of three years continued to plead with the public on X. 'Do not underestimate the power of this technology,' Coxon wrote. Jacob Coxon, a researcher for OpenAI and Anthropic, announced he quit the business through social media on Tuesday and warned that neither company is 'acting responsibly' 'These will soon be superhuman systems that can hack anything, revolutionize any field overnight, and acquire real power and resources. 'We have all witnessed the progress in each of these domains, and progress is not slowing. 'The people building AI /sciencetech/ai/index.html earnestly believe that it could kill us all by the end of the decade. This is not a marketing stunt.' He claimed that while executives and senior researchers 'couch their phrasing in the press to sound sensible,' they are privately expressing similar concerns. Coxon's comments come shortly after Geoffrey Hinton, a Canadian researcher often referred to as the 'Godfather of AI' – warned that superintelligent systems could 'lead to human extinction /sciencetech/article-16111563/Godfather-AI-superintelligent-systems-human-extinction.html .' 'We would be very foolish to develop superintelligence now, when there is no scientific consensus it can be developed safely and controllably,' Dr Hinton said. 'Losing control over AI smarter than ourselves could be catastrophic and could even lead to human extinction.' In February, Anthropic was accused of endangering national security after Amodei refused to back down over concerns the company's products could be used in ways that would violate its safeguards. Defense Secretary Pete Hegseth and other officials took to social media to chastise Anthropic for failing to allow the military unrestricted use of its AI technology earlier this year President Donald Trump slammed the AI firm at the time and said: 'We don't need it, we don't want it, and will not do business with them again ' Defense Secretary Pete Hegseth also deemed the company a 'supply chain risk,' a designation typically stamped on foreign adversaries that could derail the company's critical partnerships with other businesses. In a statement issued at the time, Anthropic said it would challenge what it called an unprecedented and legally unsound action 'never before publicly applied to an American company.' The company had said it sought narrow assurances from the Pentagon /news/the-pentagon/index.html that its AI chatbot Claude would not be used for mass surveillance of Americans or in fully autonomous weapons. The Pentagon said it was not interested in such uses and would only deploy the technology in legal ways, but it also insisted on access without any limitations. 'No amount of intimidation or punishment from the Department of War will change our position on mass domestic surveillance or fully autonomous weapons,' the company said. 'We will challenge any supply chain risk designation in court.'