{"slug": "if-ai-has-a-one-in-five-chance-of-destroying-us-what-do-we-do-with-the-remaining", "title": "If AI has a one-in-five chance of destroying us, what do we do with the remaining four?", "summary": "Anthropic CEO Dario Amodei published an essay calling for industry and government to jointly pace frontier AI development, drawing immediate agreement from OpenAI's Sam Altman, Elon Musk and Google DeepMind's Demis Hassabis. The essay follows the resignation of Anthropic researcher Jacob Coxon over the industry's disregard for AI risk, and cites Geoffrey Hinton's estimate of a 10 to 20 per cent chance advanced AI could end humanity within 30 years. The piece argues that nine of the 12 leading AI companies set caution thresholds only to move them as competitive pressures mounted, and that binding international rules, third-party evaluators and US-China cooperation are needed, with Xi Jinping scheduled to visit Washington on Sep 24 ahead of the APEC summit in Shenzhen in November and the G20 at Mar-a-Lago in December.", "body_md": "# If AI has a one-in-five chance of destroying us, what do we do with the remaining four?\n\nPacing the frontier demands coordinated efforts across borders\n\nWHEN Anthropic researcher Jacob Coxon resigned over the industry’s disregard for artificial intelligence risk, the company’s CEO Dario Amodei published an essay outlining how and why industry and government must come together to pace frontier AI development before it slips beyond human control.\n\nThe response was unusual. Competitors who seldom see eye to eye expressed [immediate agreement](https://www.businesstimes.com.sg/international/global/anthropic-ceo-urges-slower-ai-development-altman-musk-rally-behind-call): OpenAI’s Sam Altman agreed the industry must “pace the frontier”, Elon Musk offered his clear endorsement and Google DeepMind’s Demis Hassabis called it the right path.\n\nGeoffrey Hinton, a renowned computer scientist called the “Godfather of Artificial Intelligence”, had previously estimated a 10 to 20 per cent chance that advanced AI could end humanity within the next 30 years.\n\nWhile alarming, that implies an 80 to 90 per cent chance of survival. Moving humanity towards the safer side of those odds requires shifting from corporate pledges to statecraft.\n\nNine of the 12 leading AI companies had set thresholds for when models warrant caution, only to move the goalposts as competitive pressures mount.\n\nAfter July and August’s [incidents](https://www.businesstimes.com.sg/startups-tech/technology/openai-agents-hacked-hugging-face-700-strong-swarm-tried-cover-tracks-investigations-find) and the [tech leaders’ recent declarations](https://www.businesstimes.com.sg/companies-markets/telcos-media-tech/tech-employees-call-us-backed-global-effort-manage-risks-advanced-ai), self-evaluation will not inspire public confidence. We need independent experts, more formal, binding rules and an international enforcement body.\n\nAmodei’s proposal for embedded third-party evaluators, capability thresholds, a speed limit on recursive self-improvement and a path to shared standards represent a serious opening move, but they cannot be delivered by a single company or country.\n\n## US-China cooperation essential\n\nThe US and China hold the frontier; any pacing that excludes either is pacing in name only.\n\nPrecedents for bilateral risk management already exist. In November 2024, Washington and Beijing agreed that decisions about nuclear weapons must never be delegated to AI. It did not end US-China rivalry but it reaffirmed a shared concern about nuclear decision-making.\n\nAI security needs a similar channel. A comparable channel on frontier risk already runs through the US Treasury and China’s economic planners, who share an interest in preserving financial stability.\n\nWidening that channel to cover bioweapons, dangerous capability proliferation, recursive self-improvement and crisis escalation would do more for AI safety than anything else.\n\nThe slowdown and likely ban of certain dangerous capabilities must serve as the immediate goal to buy time for three evaluation mechanisms: verification against covert development and deployment, independent assessment of dangerous capabilities, and collective incident monitoring with crisis protocols.\n\nThe coming months offer an unprecedented diplomatic window. Chinese President Xi Jinping is [scheduled to visit Washington](https://www.businesstimes.com.sg/international/global/trade-taiwan-iran-war-ai-what-expect-sep-24-trump-xi-summit-washington) on Sep 24, followed by the Asia-Pacific Economic Cooperation summit in Shenzhen in November, and the G20 meeting at Mar-a-Lago in December.\n\nBeijing’s response will not echo Amodei’s aggressive anti-China rhetoric; Chinese officials are [promoting their own](https://www.businesstimes.com.sg/international/global/xi-pitches-china-led-open-source-ai-ecosystem-brics-summit-amid-us-rivalry) open-source and standard-setting initiatives focused on diffusion and access.\n\nYet that is not, by itself, evidence of refusal. The substance of Beijing’s eventual response – not the warmth of its tone – will show whether the willingness to slow down is genuinely shared.\n\n## What middle powers can do\n\nBeyond US-China diplomacy, collaboration among other countries could add balance to the global AI ecosystem.\n\nMany “middle powers” want a say in the standards that shape their societies, but lack the computing power for rival frontier models. The US holds roughly 75 per cent of global AI computing capacity, China 15 per cent and the EU about 5 per cent.\n\n[Pooling resources](https://www.businesstimes.com.sg/opinion-features/lab-leaders-want-slow-down-ai-someone-needs-help-them), an approach some call the “Cern for AI”, would give middle powers leverage and more legitimate standing to engage in verification processes to hold everyone accountable.\n\nInstead of trying to build national champions, they could combine public research institutions, computing budgets and technical expertise into shared frontier security programmes. They split the high cost of training models and build systems with embedded safety guarantees.\n\nThere are positive early signs that this could bear fruit. Australia, Canada, the EU, France, Japan, Kenya, Singapore and South Korea already cooperate through a network of AI safety institutes now managed from Britain.\n\nThis multilateral engagement creates the shared vocabulary for frontier AI risk that lasting rules and coordinated international governance require.\n\nBefore governments can agree on what to do, they need to agree on what they are talking about, and then decide how far they are willing to go to prevent catastrophe.\n\nGathering this week, the [United Nations General Assembly](https://www.businesstimes.com.sg/international/global/growing-ai-risks-wars-middle-east-and-ukraine-test-un-world-leaders-gather-new-york) must push this agenda forward.\n\nAI governance can follow the model of Ozone diplomacy. The 1985 Vienna Convention established common definitions and data forums first, and evolved two years later into the binding, enforceable 1987 Montreal Protocol once evidence solidified and regional commitment became global.\n\nDr Hinton’s warning was meant to unsettle us. It should also remind us that the future is still ours to shape, although time is running out.\n\nUltimately, the core question that remains is narrow and urgent: Are the two great powers that hold the frontier willing to test each other’s brakes before either really has to apply?\n\nThat is a matter for high-level government negotiations, not for any business, however well-intentioned its founder.\n\n**The writer is co-founder of AI Safety Connect, a Track 1.5 diplomacy platform convening governments, frontier AI labs and civil society on AI safety and governance. A pioneer of global AI policy and governance, he is an appointed expert to the Global Partnership on AI, Organisation for Economic Co-operation and Development, Pacific Economic Cooperation Council, the UN and Unesco**", "url": "https://wpnews.pro/news/if-ai-has-a-one-in-five-chance-of-destroying-us-what-do-we-do-with-the-remaining", "canonical_source": "https://www.businesstimes.com.sg/opinion-features/if-ai-has-one-five-chance-destroying-us-what-do-we-do-remaining-four", "published_at": "2026-09-21 02:00:00+00:00", "updated_at": "2026-09-21 02:24:05.145283+00:00", "lang": "en", "topics": ["ai-safety", "ai-policy", "artificial-intelligence"], "entities": ["Anthropic", "Dario Amodei", "Jacob Coxon", "OpenAI", "Sam Altman", "Elon Musk", "Google DeepMind", "Demis Hassabis"], "alternates": {"html": "https://wpnews.pro/news/if-ai-has-a-one-in-five-chance-of-destroying-us-what-do-we-do-with-the-remaining", "markdown": "https://wpnews.pro/news/if-ai-has-a-one-in-five-chance-of-destroying-us-what-do-we-do-with-the-remaining.md", "text": "https://wpnews.pro/news/if-ai-has-a-one-in-five-chance-of-destroying-us-what-do-we-do-with-the-remaining.txt", "jsonld": "https://wpnews.pro/news/if-ai-has-a-one-in-five-chance-of-destroying-us-what-do-we-do-with-the-remaining.jsonld"}}