cd /news/ai-safety/preprint-warns-of-catastrophic-ai-ri… · home topics ai-safety article
[ARTICLE · art-22373] src=news.uq.edu.au pub= topic=ai-safety verified=true sentiment=↓ negative

Preprint warns of catastrophic AI risks if no action is taken within five years

A survey of 272 AI experts from The University of Queensland and MIT FutureTech found a 10% probability of catastrophic AI outcomes within five years, including mass casualties, financial losses, or democratic collapse. The experts identified AI cyberattacks, weapons development, and governance failures as top risks, warning that 18 of 24 risk categories could cause catastrophic harm without immediate action. The findings call for urgent regulatory measures to prevent what researchers describe as intolerable threats comparable to nuclear power or aviation failures.

read3 min publishedJun 5, 2026

Artificial intelligence (AI) is a severe threat to national security, the financial sector and the information technology sector, according to a study of international AI risk experts.

Researchers from The University of Queensland and MIT FutureTech surveyed 272 AI experts from across the AI industry, academia, government and the community.

Associate Professor Michael Noetel from UQ’s School of Psychology said the experts judged there to be at least a 10 per cent probability of catastrophic outcomes from AI over the next 5 years.

“They assessed AI risk across 24 categories by answering 3 questions: which AI risks are most severe, who is most vulnerable to them, and who is most responsible for addressing them,” Dr Noetel said.

“The experts said the top priorities should be AI cyberattacks, weapons development, competitive pressures in AI development and governance failure.

“If no action is taken to reduce risk over the next 5 years, experts believe 18 of the 24 risks are more than 10 per cent likely to cause catastrophic outcomes.

“Unless something changes, this is at least a 10 per cent chance of more than one million deaths, more than USD $100 billion in financial losses, or global collapse of democracy and civil rights.”

The experts were asked to assess the probabilities of 5 levels of harmful outcomes occurring, from minor to catastrophic harms.

The harms included physical, financial and societal harms such as authoritarian takeover.

The research found that even if pragmatic mitigations were adopted over the next 5 years, 5 of the 24 risk categories were estimated more than 10 per cent likely to cause catastrophic outcomes.

Dr Neil Thompson, Director of MIT FutureTech, said the expert estimations were incredibly worrying.

"If we consider other mature areas of technology such as nuclear power or aviation, risks at that level would be treated as intolerable,” Dr Thompson said.

“These findings are a call to arms for those building and governing AI to make sure the worst outcomes do not occur.”

The findings suggest many sectors are exposed to overlapping risks as AI becomes embedded in decision-critical systems.

The experts also expressed concern that even without catastrophic outcomes, many AI risks such as false and misleading information, overreliance on AI systems and fraud and scams, could reshape business and society.

Dr Peter Slattery from MIT FutureTech said while the public and users of AI models bear the brunt of AI harm, they have limited power to mitigate them.

“And because AI development is highly competitive, labs and governments have limited incentives to slow down or invest sufficiently in safety,” Dr Slattery said.

“This is why laws, treaties and other collective-action mechanisms may become necessary.”

The researchers used the Delphi method to collect 3 rounds of expert contributions anonymously. Answers were de-identified and reported in aggregate, limiting the potential for individual interests to influence results.

Collaboration and acknowledgements #

The research was co-authored by Dr Alexander Saeri, Jess Graham and Associate Professor Michael Noetel from The University of Queensland and Dr Peter Slattery and Dr Neil Thompson from MIT FutureTech.

[Three new Ebola vaccines are being developed. An infectious disease expert explains](/2026-06-three-new-ebola-vaccines-are-being-developed-infectious-disease-expert-explains)

[Gestational diabetes shares strong genetic links with type 2 diabetes](/2026-06-gestational-diabetes-shares-strong-genetic-links-type-2-diabetes)

Media contact

UQ Communicationscommunications@uq.edu.au[+61 429 056 139](tel:+61 429 056 139)

── more in #ai-safety 4 stories · sorted by recency
sponsored brought to you by zahid.host 4,200+ EU-deployed projects
reading about agents? ship yours in a single git push.

Run your AI side-project on zahid.host

EU-based hosting, git-push deploys, automatic HTTPS, no cold starts. Free tier with a custom domain — perfect for shipping the agent you just read about.

$git push zahid main
Live at https://your-agent.zahid.host
Get free account → Pricing
from €0/mo · no card required
LIVE [news/preprint-warns-of-ca…] indexed:0 read:3min 2026-06-05 ·