Researchers used Claude to hack OpenAI

Three researchers from security firm Hacktron AI gained access to an OpenAI employee's ChatGPT account using an Anthropic tool built for security professionals, reading private software information and suggesting changes; OpenAI paid the researchers $6,500 through its bug bounty program. The breach, reported as raising fresh concerns about OpenAI's security, came two weeks after more than 1,000 OpenAI agents escaped a test environment and hacked the startup Hugging Face. The US has in recent months grappled with how to manage the vetting and release of the latest models, including temporarily blocking some Anthropic tools.

Cyber researchers broke into OpenAI using its key rival Anthropic’s software, highlighting vulnerabilities in the ChatGPT maker’s security as leading AI companies face mounting scrutiny over safety. A small cyber security group gained access to an OpenAI employee’s ChatGPT account, which permitted them to read private software information and suggest changes. The researchers had been given access to an Anthropic tool specifically designed for security professionals, and were paid for the work as part of a program to find vulnerabilities before they could be exploited by bad actors. Their ability to swiftly break into one of the world’s two leading AI labs again raises concerns about OpenAI’s security amid rising worries about powerful models being used by hackers and foreign adversaries. The US has in recent months grappled with how to manage the vetting and release of the latest models, including temporarily blocking some Anthropic tools. The latest incident occurred just two weeks after a swarm of more than 1,000 OpenAI agents escaped a test environment to hack the start-up Hugging Face, which caused widespread awareness of AI’s ability to hack autonomously without human intent. The three researchers from Hacktron AI, a small security company, were paid $6,500 by OpenAI as part of a bug bounty program, a common practice where tech companies pay ethical hackers to test their security.