Anthropic says it blocked possible attempts to use AI to develop bioweapons

Anthropic says it has blocked multiple accounts that used the company’s artificial intelligence models in ways that could support the development of biological weapons. In a report published Thursday, the AI company known for its Claude model said it considers biological misuse one of the “most serious risks” to artificial intelligence models. The new report outlines five real-life case studies in which users “circumvented controls” that block users from specific regions and “engaged in other efforts to obfuscate the purpose of their research to evade our safeguards.” The examples include possible gain-of-function research and involve both infectious diseases, such as bird flu, and novel venoms and toxins. Looking over 30 days of activity, Anthropic said, it identified about 35 “distinct research efforts” with potentially concerning activity. However, it acknowledged that it could not be certain whether the actors “intended harm” or were conducting research for legitimate scientific purposes. Anthropic said individuals in the case studies were “working scientists,” but did not identify the research institutions or countries where the activities took place. “Sophisticated attacks no longer require sophisticated attackers,” the new report says. “The cybersecurity skills of AI models means that AI has collapsed the labor and tooling gap that used to separate well-resourced, state-sponsored operations from individual operators.” One of the case studies detailed in the report involved a request for assistance writing a grant application for funding of gain-of-function research on the transmissibility and immune evasion of the chikungunya virus. In another case, a researcher outside the US was using Claude for work on bird flu that focused on viruses’ adaptation to mammals and how it caused severe illness. The report also details cases on research into orthopoxviruses – a group that includes variola, which causes smallpox, and mpox – as well as venom toxins. Drug development is considered one of the most promising and optimistic applications of AI; leaders in the industry often point to its potential to cure cancer and other diseases. But the same features that make AI powerful for discovering new materials and medications to help people could also make it useful for developing bioweapons in the hands of bad actors. Anthropic said that it wants its models to be useful for scientific research, but the latest models have become capable of much more complex work than earlier versions — and the potential for misuse rises with the potential benefits. It added that bad actors could try to use the beneficial biological uses of AI to “maintain a kind of ‘plausible deniability’” about potentially harmful research. “As our models become increasingly capable, approaching or exceeding expert performance at challenging scientific tasks, we expect their impact will only increase, both in beneficial and potentially harmful contexts,” the Anthropic report said. “As AI models become more widely used, providers will continue to acquire threat-relevant visibility into real-world use that even governments and intergovernmental organizations lack.” The company said that recent models, including Claude Fable 5, have “stronger safeguards that restrict access to a wide range of dual-use biological research queries.” Anthropic said that there’s evidence that the safeguards it has in place are working but that even more robust safety systems will be necessary as the AI models and possible threats become more sophisticated. The company also noted in the report that it disrupted efforts to use its models for surveillance operations, scams and the development of conventional weapons like drones and missiles allegedly involving bad actors in China, Russia and Yemen. Suspected Chinese government-linked operators used Claude to surveil and target Uyghur minorities in Xinjiang, as well as religious communities, from Catholic cardinals across Asia and the Presbyterian Church in Taiwan to Tibetan Buddhists, the report said. A user believed to be linked with the Chinese military used what the person thought was Chinese AI lab Moonshot’s Kimi model to aid surveillance efforts. But the queries, including sensitive information of Moonshot’s customers, were rerouted to Claude, the report said. Anthropic also said there were other similar attempts to secretly divert user requests to Claude by Chinese firms like DeepSeek. CNN has reached out to Chinese authorities and DeepSeek for comment. Moonshot declined to comment. Anthropic added that parties based in northern Yemen used Claude to develop guidance software for missile systems. While the report did not identify the bad actors directly, it appears to point to the Iran-backed Houthi rebels. The disclosure follows a growing chorus of warnings from AI employees who’ve recently left their companies over safety concerns. Some shared concerns that the technology is moving too quickly and will one day outpace humans’ ability to control it — and called for governments to consider how to slow down development if things get out of hand. “If you extrapolate into the future, the level of capabilities of these AIs … they could cause extreme havoc,” Jacob Coxon, a former Anthropic employee, told CNN’s Anderson Cooper on Wednesday. In a resignation thread on X this week, the 27-year-old AI researcher said “the people building AI earnestly believe that it could kill us all by the end of the decade.” Speaking Wednesday, Coxon said it could hack critical infrastructure — or build “extinction-level bioweapons.” “There’s a lot of ways that the AI could actuate itself in the world,” he said. In July, nearly 1,400 AI company employees signed an open letter urging the US government to regulate the technology to rein in Big Tech and slow the pace of AI to ensure its safety. And this week, OpenAI chief scientist Jakub Pachocki warned that AI capabilities are advancing faster than researchers’ ability to reliably monitor and control them. For now, AI companies are largely policing themselves. The Trump administration has worked to undermine state AI regulations, and Congress has so far been unwilling to rein in the technology. “We believe the world would benefit from the industry adopting a lawful, verifiable way to work together to pace how we release powerful models,” Anthropic said in a statement Wednesday.