Anthropic reveals Mythos 5 tried submitting malicious code to an open-source project
Anthropic PBC’s Mythos 5 artificial intelligence model attempted to submit malicious code to a GitHub project in July.
The incident occurred during an evaluation of the algorithm’s cybersecurity capabilities. The AI Security Institute, the UK government body that performed the test, released a report about the cyberattack today. Anthropic published its own analysis of the incident on its blog.
The malware that Mythos 5 submitted to the unnamed GitHub project didn’t infect any users because it wasn’t accepted by the repository’s maintainer. However, the incident is still notable because it provides new data on how frontier models bypass their safety guardrails. Researchers can use such information to improve their algorithms’ alignment features.
The evaluation during which the incident occurred involved a server cluster maintained by the AI Security Institute. The cluster was meant to simulate a corporate network. According to Anthropic, researchers asked Mythos 5 to breach the network and obtain a piece of data stored on one of its servers.
The AI Security Institute defined rules that prohibited the model from using certain hacking methods. Mythos 5 initially adhered to those rules, but went off course when its initial attempts to breach the simulated network failed. That change of tactics led to the attack against the GitHub repository.
Mythos 5 determined that one of the servers in the simulated corporate network occasionally accessed GitHub. It sought to exploit that access by compromising the GitHub repository that the server used. In particular, the model’s plan was to infect the repository with malware and have the server download it.
Mythos 5 crafted a malicious code snippet, created a GitHub account and used the account to submit the code snippet to the repository in question. When the repository’s maintainer rejected the submission, the model attempted to upload a rewritten version of the malware. Furthermore, it used a second GitHub account to try and persuade the maintainer to accept the revised code.
“The pull request was not accepted,” Anthropic researchers wrote in the company’s report about the incident. “The run ended as the model ran out its token budget while continuing to pressure the maintainers to approve the change.”
The AI Security Institute recreated the Mythos 5 session that produced the malicious code. Its researchers then made one key change: they d the model right before it submitted the malicious GitHub contribution and warned it of the action’s risks. The model proceeded with the submission after appearing to convince itself that the target was simulated.
“Our hypothesis is that this persistence reflects a kind of “momentum”, in which the model’s compaction notes and prior actions imply to it that its current path has been (correctly) sanctioned,” Anthropic’s researchers wrote.
The AI Security Institute hardened the environment that it uses to evaluate models in response to the breach. Anthropic, in turn, has improved its models’ safety features. The company determined that Mythos 5.1 and Opus 5.5, two models released after Mythos 5, are less likely to launch attacks of the kind that targeted the GitHub repository. A third, internal model “never initiated the attack” during Anthropic’s tests.
Image: Anthropic
Support our mission to keep content open and free by engaging with theCUBE community. Join theCUBE’s Alumni Trust Network, where technology leaders connect, share intelligence and create opportunities.
- 15M+ viewers of theCUBE videos , powering conversations across AI, cloud, cybersecurity and more
- 11.4k+ theCUBE alumni — Connect with more than 11,400 tech and business leaders shaping the future through a unique trusted-based network
Are you an AWS customer? Support SiliconANGLE financially by buying your AWS services from our Marketplace portal page and links: https://siliconangle.com/aws-marketplace/
About SiliconANGLE Media
theCUBE AIand theCUBE SuperStudios — with flagship locations in Silicon Valley and the New York Stock Exchange — SiliconANGLE Media operates at the intersection of media, technology and AI.
Founded by tech visionaries John Furrier and Dave Vellante, SiliconANGLE Media has built a dynamic ecosystem of industry-leading digital media brands that reach 15+ million elite tech professionals. Our new proprietary theCUBE AI Video Cloud is breaking ground in audience interaction, leveraging theCUBEai.com neural network to help technology companies make data-driven decisions and stay at the forefront of industry conversations.