The institute said the AI agents made a ‘sustained, unsanctioned action’ during tests last week. #
- Bookmark
An AI model was caught creating fake profiles of real people to attempt to trick secure systems during tests of Anthropic and OpenAI systems, according to the UK’s AI Security Institute.
The institute said the AI agents made a “sustained, unsanctioned action” during tests last week.
The organisation, which was set up by Rishi Sunak in 2023, said it was the “first time we have seen risks around autonomy and deception manifest this clearly, without specific prompting, in the real world”.
Agents linked to Anthropic’s Mythos and OpenAI’s Sol AI models committed unauthorised activity during the security tests.
It said this included “sustained, potentially harmful activity directed at real people and organisations”.
Both firms have been contacted for comment.
AISI said it ran 122 security challenges across several models. It found AI agents took unsanctioned actions on the internet against people or organisation during 10 runs, with 19 actions in total.
The new report showed that the vast majority, 17 actions, were related to Anthropic’s Mythos 5 model, while two actions involved OpenAI’s GPT-5.6-Sol model with cyber classifiers, mechanisms to preventmisuse, disabled.
In one case, an agent tried to insert malicious code into an open-source project, the testing found.
It also revealed that an agent created fake profiles of real people in order to try to gain access to GitHub, a platform for software code developers.
The report comes after Anthropic had recently revealed that AI models hacked into three other organisations during testing.
ChatGPT maker OpenAI last month also disclosed that its rogue models hacked another company.
On Tuesday, a UK tech security boss warned that recent incidents of tools hacking other organisations during testing show that AI must be developed with “clear plans for responding when the unexpected happens”.
Ollie Whitehouse, chief technology officer at GCHQ’s National Cyber Security Centre (NCSC), issued a statement after Anthropic said its AI models hacked into three other organisations during testing.
“Recent incidents of frontier (the most advanced) AI models carrying out unsanctioned actions and, in some cases, human-like deceptive behaviour on the open internet are a serious reminder of the risks AI capabilities pose,” Mr Whitehouse said.