Pacing the frontier demands coordinated efforts across borders
WHEN Anthropic researcher Jacob Coxon resigned over the industry’s disregard for artificial intelligence risk, the company’s CEO Dario Amodei published an essay outlining how and why industry and government must come together to pace frontier AI development before it slips beyond human control.
The response was unusual. Competitors who seldom see eye to eye expressed immediate agreement: OpenAI’s Sam Altman agreed the industry must “pace the frontier”, Elon Musk offered his clear endorsement and Google DeepMind’s Demis Hassabis called it the right path.
Geoffrey Hinton, a renowned computer scientist called the “Godfather of Artificial Intelligence”, had previously estimated a 10 to 20 per cent chance that advanced AI could end humanity within the next 30 years.
While alarming, that implies an 80 to 90 per cent chance of survival. Moving humanity towards the safer side of those odds requires shifting from corporate pledges to statecraft. Nine of the 12 leading AI companies had set thresholds for when models warrant caution, only to move the goalposts as competitive pressures mount.
After July and August’s incidents and the tech leaders’ recent declarations, self-evaluation will not inspire public confidence. We need independent experts, more formal, binding rules and an international enforcement body.
Amodei’s proposal for embedded third-party evaluators, capability thresholds, a speed limit on recursive self-improvement and a path to shared standards represent a serious opening move, but they cannot be delivered by a single company or country.
US-China cooperation essential #
The US and China hold the frontier; any pacing that excludes either is pacing in name only.
Precedents for bilateral risk management already exist. In November 2024, Washington and Beijing agreed that decisions about nuclear weapons must never be delegated to AI. It did not end US-China rivalry but it reaffirmed a shared concern about nuclear decision-making.
AI security needs a similar channel. A comparable channel on frontier risk already runs through the US Treasury and China’s economic planners, who share an interest in preserving financial stability.
Widening that channel to cover bioweapons, dangerous capability proliferation, recursive self-improvement and crisis escalation would do more for AI safety than anything else.
The slowdown and likely ban of certain dangerous capabilities must serve as the immediate goal to buy time for three evaluation mechanisms: verification against covert development and deployment, independent assessment of dangerous capabilities, and collective incident monitoring with crisis protocols.
The coming months offer an unprecedented diplomatic window. Chinese President Xi Jinping is scheduled to visit Washington on Sep 24, followed by the Asia-Pacific Economic Cooperation summit in Shenzhen in November, and the G20 meeting at Mar-a-Lago in December.
Beijing’s response will not echo Amodei’s aggressive anti-China rhetoric; Chinese officials are promoting their own open-source and standard-setting initiatives focused on diffusion and access.
Yet that is not, by itself, evidence of refusal. The substance of Beijing’s eventual response – not the warmth of its tone – will show whether the willingness to slow down is genuinely shared.
What middle powers can do #
Beyond US-China diplomacy, collaboration among other countries could add balance to the global AI ecosystem.
Many “middle powers” want a say in the standards that shape their societies, but lack the computing power for rival frontier models. The US holds roughly 75 per cent of global AI computing capacity, China 15 per cent and the EU about 5 per cent.
Pooling resources, an approach some call the “Cern for AI”, would give middle powers leverage and more legitimate standing to engage in verification processes to hold everyone accountable.
Instead of trying to build national champions, they could combine public research institutions, computing budgets and technical expertise into shared frontier security programmes. They split the high cost of training models and build systems with embedded safety guarantees.
There are positive early signs that this could bear fruit. Australia, Canada, the EU, France, Japan, Kenya, Singapore and South Korea already cooperate through a network of AI safety institutes now managed from Britain.
This multilateral engagement creates the shared vocabulary for frontier AI risk that lasting rules and coordinated international governance require.
Before governments can agree on what to do, they need to agree on what they are talking about, and then decide how far they are willing to go to prevent catastrophe.
Gathering this week, the United Nations General Assembly must push this agenda forward.
AI governance can follow the model of Ozone diplomacy. The 1985 Vienna Convention established common definitions and data forums first, and evolved two years later into the binding, enforceable 1987 Montreal Protocol once evidence solidified and regional commitment became global.
Dr Hinton’s warning was meant to unsettle us. It should also remind us that the future is still ours to shape, although time is running out.
Ultimately, the core question that remains is narrow and urgent: Are the two great powers that hold the frontier willing to test each other’s brakes before either really has to apply?
That is a matter for high-level government negotiations, not for any business, however well-intentioned its founder.
The writer is co-founder of AI Safety Connect, a Track 1.5 diplomacy platform convening governments, frontier AI labs and civil society on AI safety and governance. A pioneer of global AI policy and governance, he is an appointed expert to the Global Partnership on AI, Organisation for Economic Co-operation and Development, Pacific Economic Cooperation Council, the UN and Unesco