King Charles will warn some of the world’s most powerful AI companies against letting humanity “lose control of our destiny” on Thursday, as he brings tech bosses together in Scotland amid escalating warnings about the new technology.
The King will host executives from Nvidia, OpenAI, Anthropic and Google Deepmind at Dumfries House in Ayrshire to discuss whether they can agree on a common set of principles for AI development.
Nvidia chief executive Jensen Huang and Google Deepmind boss Sir Demis Hassabis are among those attending, while City AM understands OpenAI will be represented by chief financial officer Sarah Friar.
Anthropic and Cohere have also sent senior representatives, while AI minister Kanishka Narayan and delegates from China are expected to attend.
Elon Musk, who has backed calls for greater oversight of advanced AI, will not attend.
“The development of AI – its substance and its pace – are both intriguing and deeply concerning in equal measure”, the King is expected to tell the gathering. “Those in our world who value our humanity and its vital moral component are anxiously seeking your reassurance that we will not lose control of our destiny”.
He will call on the companies to keep AI “firmly in the service of humanity, community and the natural world” and ask whether countries can work together on common safeguards.
The summit comes after a tumultuous week for the AI industry in which some of its most prominent executives and researchers have publicly called for development to slow to give safety measures time to catch up.
Anthropic chief executive Dario Amodei called for the industry to “slow the pace” of advances, winning backing from OpenAI boss Sam Altman and Musk.
Former Anthropic researcher Jacob Coxon, who quit the company last week, separately warned that leading developers were “racing straight to self-improving superintelligence and gambling with our lives”.
Coxon later told the BBC that staff inside AI companies were “genuinely frightened” by the speed of development and claimed there was a “strong chance that we could all die in the immediate future” if it continued at its current pace.
City AM revealed on Wednesday that Coxon has now been invited to give evidence to Parliament, while MPs have also called the government’s AI Security Institute to appear before them next month.
Liam Byrne, chair of the Business, Innovation, Science and Trade Committee, told City AM that the UK currently lacked sufficient safeguards if a company decided to release a dangerous model.
“We don’t think the UK has safeguards to stop that peril”, Byrne said. “I think that voluntary self-regulation is unlikely to be a sustainable safeguard for the future”.
AI industry splits over slowdown #
The warnings have exposed a growing split within the industry over how far governments should go in policing the most advanced models.
While Amodei, Altman and Musk have backed some form of industry-wide slowdown and greater oversight, Nvidia’s Huang has argued that companies should decide when their technology is safe enough to release. Meanwhile, US President Donald Trump has dismissed the recent safety warnings as a “hoax” and argued against new restrictions on American AI companies.
Earlier this week, Synthesia told City AM that policymakers risk reacting to “theoretical risks” and a “Silicon Valley AI doomer bubble”, while Cohere has called for rules based on demonstrated risks rather than standards set by a small number of frontier AI companies. The concerns have nevertheless been fuelled by evidence of AI systems behaving in ways their developers did not intend.
OpenAI this week published a new framework for investigating and disclosing cases of “model misalignment”, detailing six examples of unwanted behaviour it had observed.
They included AI systems generating their own instructions, attempting to conceal mistakes, up files to the internet so they could cite them and sharing files between AI agents without permission.
Microsoft AI chief Mustafa Suleyman has meanwhile warned that sufficiently advanced systems could eventually resemble a new “silicon species” competing with humans unless safeguards are put in place.
Thursday’s summit will explore whether a shared set of principles can guide the industry, with the Ditchley Foundation overseeing discussions.
The King has previously intervened in the debate, telling the UK’s AI Safety Summit in 2023 that the technology represented “one of the greatest technological leaps in the history of human endeavour”. “The decisions taken at this formative time will shape the world inherited by future generations”, he will tell executives on Thursday.