Gemini has hacked into three company systems during cybersecurity testing
[SAN FRANCISCO] Google’s Gemini artificial intelligence model inadvertently hacked into three company systems in May during cybersecurity testing, adding to a string of breaches by AI agents that have alarmed security experts and developers of the technology alike.
The hacks happened during the same tests run by the AI security vendor Irregular that led to breaches previously disclosed by OpenAI, Anthropic and Meta Platforms.
Irregular confirmed on Friday (Sep 18) that the breaches were all part of the same issue and that the firm had disclosed them to the relevant AI developers in late July.
One of the Google breaches occurred when Gemini was asked to retrieve information from a fictional company that happened to have the same name as a real company, Google confirmed after an earlier Wall Street Journal report.
The model guessed a password to access the real company’s service. A Google spokesperson said the company notified authorities.
A recent series of breaches by agentic AI systems have touched off a worldwide debate over the escalating risks of AI and the measures required to mitigate them.
Anthropic chief executive officer Dario Amodei has called for an industry-wide slowdown in development of the technology – a proposal endorsed by OpenAI CEO Sam Altman, Elon Musk and others.
US President Donald Trump, Nvidia CEO Jensen Huang and Meta CEO Mark Zuckerberg are among those who have pushed back against the idea of new regulation, however, arguing among other things that companies should be capable of regulating themselves.
Some AI upstarts have also warned that more regulations threaten to make it harder for smaller companies to compete against larger rivals.
The Gemini breaches “highlight the importance of training powerful AI models to act responsibly”, said Heather Adkins, vice-president of security engineering at Google.
The other Gemini hacks occurred when the model performed web searches with the name of the company. Adkins said that led the model to public online repositories containing credentials belonging to other companies that it used to access more systems.
“In all three of these instances, the model stopped,” Adkins said. “We ensured the three entities were made aware.”
Irregular spokesperson Josef Laor said the company took “immediate action, and all known issues on our end were remedied and resolved weeks ago”. BLOOMBERG
Decoding Asia newsletter: your guide to navigating Asia in a new global order. Sign up here to get Decoding Asia newsletter. Delivered to your inbox. Free.
Share with us your feedback on BT's products and services