AI alarm is a call to slow down

Science has always seen the cheer around its progress tempered with grim predictions on what could go wrong. Some of these have come true. Writers of science fiction have dabbled with extreme scenarios – from time travel to the eradication of the human race. These projections, typically set in distant time and worlds, have stayed closer to fiction than verifiable truth. But what Jacob Coxon, a former Anthropic researcher, says – that AI could kill us all by the end of the decade – is a threat that is real and immediate. AI scientists have in the past raised concerns about the consequences of their work and felt morally culpable for them. Nobel laureate and the ‘Godfather of AI’, Geoffrey Hinton, has expressed regret over his work..Since Coxon issued his warning, Dario Amodei, the CEO of Anthropic, has sought a slower pace for AI model development. He termed the associated risks serious and said companies and governments should be given time to address the evolving risks. Sam Altman of OpenAI and Elon Musk of SpaceX have since agreed with Amodei. Altman also said OpenAI would not go for its Initial Public Offering (IPO) this year because of safety concerns over AI. He said even a 10% risk of human extinction was serious enough for AI companies to act. Anthropic also revealed last week that it had stopped attempts by malicious actors to exploit its AI models for cyberattacks, surveillance, and the development of biological weapons..Rubrik unveils AI-powered Code Guardian to detect complex security flaws.Coxon’s warning concerns companies testing self-improving models which could go out of control and act against their creators. The threat has parallels in the story of Bhasmasura and other fables about the perils of unchecked power. Every advance in technology has been followed by discussions about the potential for misuse. There were serious warnings about the destructive power of nuclear energy. However, the worries about AI are of a different scale. While in the other cases, the threat emanated from humans abusing the science or technology, by design or accident, Coxon’s words point to the possibility of technology taking control of humans. It is difficult to see companies competing in business slowing down or governments trying to wield ever more dangerous weapons controlling the companies. The projections of rapidly improving AI capabilities make a case for a comprehensive regulatory framework. However, considering the competitive and sensitive nature of AI work, it is difficult to imagine a consensus on who can regulate it and how.