A frog does not know it is being boiled until it is too late. Lawmakers in London feel the same about artificial general intelligence—the next chapter of the 21st century technology revolution which could be as little as three years away.
A campaign for government recognition that such non-human “superintelligence” should be formally considered a threat to national and global security is backed by more than 125 U.K. lawmakers. The philosopher, historian and author of mega-selling books, Yuval Noah Harari, has given the campaign his personal endorsement.
Luciana Berger, the former Labour MP and now a member of the upper legislative house in the U.K., the House of Lords, is leading the charge via the campaign group ControlAI. “On a domestic level, we are being left behind when it comes to AI development,” she told me. “As a country, the U.K. is completely reliant on American and Chinese AI models. We are a country renowned for our research and development, this needs to be right at the top of our agenda in order to maintain even a tiny stake in AI development.”
Berger speaks for many political leaders who are becoming increasingly trepidatious about the development of AI. Fears over digital sovereignty are growing and at present there are few checks and balances for a new technology that Demis Hassabis of DeepMind has described as “nothing less than the dawning of a new age for humanity.”
Controls, such as they are, are largely in the hands of the firms that are developing the technology, an obvious weakness in an age when moving fast and breaking things appears to be back in vogue.
“On a global level, we face a far more existential risk,” Berger said. “Leading AI companies are racing towards superintelligence, despite acknowledging that it could pose an existential threat to humanity. We have no idea when these systems could come into play or the extent to which they will disrupt the global order.”
Hassabis has called for “robust safeguards” and a new regulatory regime led by America. Harari has gone further, arguing that even the ill-fated Titanic did not build the iceberg it eventually ploughed into. He has called for an equivalent of the US Food and Drug Administration to control new AI tools.
The public can be forgiven for nodding in agreement. Anthropic, a leading AI firm, said one of its cybersecurity products, Mythos, was too dangerous to release for general use. Last week, OpenAI revealed that an autonomous agent it was testing had “escaped” a secure area and attacked the AI startup Hugging Face. A few days later, Reuters revealed that another firm, Modal, was aware of attempts by the same rogue agent to compromise users of its platforms.
OpenAI’s statement will have done little to soothe concerns. It read: “Last week, Hugging Face disclosed a new kind of security incident after it detected and contained an AI agent that compromised their infrastructure, something we expect to become more commonplace with the proliferation of increasingly cyber-capable models.” It was reported that the agent was outside its test area for a week before OpenAI realized. By then the incident had already been reported to the FBI.
“When the OpenAI cyber-attack happened last week, I could not believe how far down I had to scroll through my news to see reports that it had happened,” Berger said. “Dangers like this should be front and center of all our discourse, leading to urgent action.
“There is a definite need for regulation, but this is not just a U.K. issue. Whilst, yes, global regulation would be difficult, it must urgently be the goal. AI is like no other policy challenge.”
*Read more: All hail London—whatever the U.K.’s new Prime Minister says *
The European Union’s AI Act is one of the first attempts at pan-national controls of AI implementation. ControlAI is active in Germany, Canada, and the U.S., where the bi-partisan AI Kill Switch Act has just been introduced. Public support is high.
“It is significant that the new U.K. Prime Minister has prioritized this area, appointing an AI Minister who will now attend Cabinet,” Berger said. “Success for me is serious and significant investment in U.K. sovereign AI capabilities and superintelligent AI to be formally recognized as a national and global security threat, with guardrails put in place.”
AI brings remarkable opportunities for advancement—whether it’s solving health challenges, better understanding climate change, or cleaning up email inboxes. “We’ve essentially found a way to make sand think,” Hassabis said. The worry is that nobody is yet across what that actually means.
Fortune CEO Forum convenes Europe's top CEOs in London for private conversations on AI, growth and leadership as well as exclusive insights from the Fortune 500 Europe list.