Preprint warns of catastrophic AI risks if no action is taken within five years

A survey of 272 AI experts from The University of Queensland and MIT FutureTech found a 10% probability of catastrophic AI outcomes within five years, including mass casualties, financial losses, or democratic collapse. The experts identified AI cyberattacks, weapons development, and governance failures as top risks, warning that 18 of 24 risk categories could cause catastrophic harm without immediate action. The findings call for urgent regulatory measures to prevent what researchers describe as intolerable threats comparable to nuclear power or aviation failures.

Global experts assess risk of AI catastrophes Artificial intelligence AI is a severe threat to national security, the financial sector and the information technology sector, according to a study of international AI risk experts. Researchers from The University of Queensland and MIT FutureTech surveyed 272 AI experts from across the AI industry, academia, government and the community. Associate Professor Michael Noetel https://about.uq.edu.au/experts/37507 from UQ’s School of Psychology https://psychology.uq.edu.au/ said the experts judged there to be at least a 10 per cent probability of catastrophic outcomes from AI over the next 5 years. “They assessed AI risk across 24 categories by answering 3 questions: which AI risks are most severe, who is most vulnerable to them, and who is most responsible for addressing them,” Dr Noetel said. “The experts said the top priorities should be AI cyberattacks, weapons development, competitive pressures in AI development and governance failure. “If no action is taken to reduce risk over the next 5 years, experts believe 18 of the 24 risks are more than 10 per cent likely to cause catastrophic outcomes. “Unless something changes, this is at least a 10 per cent chance of more than one million deaths, more than USD $100 billion in financial losses, or global collapse of democracy and civil rights.” The experts were asked to assess the probabilities of 5 levels of harmful outcomes occurring, from minor to catastrophic harms. The harms included physical, financial and societal harms such as authoritarian takeover. The research found that even if pragmatic mitigations were adopted over the next 5 years, 5 of the 24 risk categories were estimated more than 10 per cent likely to cause catastrophic outcomes. Dr Neil Thompson, Director of MIT FutureTech, said the expert estimations were incredibly worrying. "If we consider other mature areas of technology such as nuclear power or aviation, risks at that level would be treated as intolerable,” Dr Thompson said. “These findings are a call to arms for those building and governing AI to make sure the worst outcomes do not occur.” The findings suggest many sectors are exposed to overlapping risks as AI becomes embedded in decision-critical systems. The experts also expressed concern that even without catastrophic outcomes, many AI risks such as false and misleading information, overreliance on AI systems and fraud and scams, could reshape business and society. Dr Peter Slattery from MIT FutureTech said while the public and users of AI models bear the brunt of AI harm, they have limited power to mitigate them. “And because AI development is highly competitive, labs and governments have limited incentives to slow down or invest sufficiently in safety,” Dr Slattery said. “This is why laws, treaties and other collective-action mechanisms may become necessary.” The researchers used the Delphi method to collect 3 rounds of expert contributions anonymously. Answers were de-identified and reported in aggregate, limiting the potential for individual interests to influence results. Collaboration and acknowledgements The research was co-authored by Dr Alexander Saeri https://psychology.uq.edu.au/profile/3283/alexander-saeri , Jess Graham https://psychology.uq.edu.au/profile/13841/jess-graham and Associate Professor Michael Noetel https://about.uq.edu.au/experts/37507 from The University of Queensland and Dr Peter Slattery https://futuretech.mit.edu/team/peter-slattery and Dr Neil Thompson https://futuretech.mit.edu/team/neil-thompson from MIT FutureTech. Related articles Three new Ebola vaccines are being developed. An infectious disease expert explains /2026-06-three-new-ebola-vaccines-are-being-developed-infectious-disease-expert-explains Gestational diabetes shares strong genetic links with type 2 diabetes /2026-06-gestational-diabetes-shares-strong-genetic-links-type-2-diabetes Media contact UQ Communications communications@uq.edu.au mailto:communications@uq.edu.au +61 429 056 139 tel:+61 429 056 139