Earlier this month, Anthropic reported five cases in which people used its models for work that could support biological weapons development. The cases included help preparing a proposal for gain-of-function research on a mosquito-borne virus at a military institute and planning experiments intended to help bird flu get better at infecting mammals. Anthropic had no way to determine the intent behind this work but blocked it anyway.
That ambiguity is the problem. The same tools that scientists might use to design a vaccine can help design a worse pathogen. The same tools that help scientists design a drug that targets cancer cells while sparing healthy ones can be tweaked to build a drug that targets healthy cells alone. There is no need for someone to ask Claude to “build a bioweapon.” They only need to look like scientists who are working on important biological problems.
Anthropic, which shut the accounts down, has been the most public about the risks of AI to the development of bioweapons. OpenAI has also put in guardrails. Other AI companies, especially with open-weight models — those whose underlying code anyone can download and modify without any company oversight — generally release their products with far fewer safeguards and little monitoring for how their tools are being used.
Much of the industry’s safety debate is about whether to “pace the frontier” — slowing model development so guardrails and defenses can catch up. That’s worth doing, but it only works if it is a bridge to building biodefense. And slowing model development certainly has no impact on who can order synthetic DNA —or what national governments are doing to build up their capabilities.
For decades, biological weapons were rare for a reason: Building them takes expertise that’s hard to acquire and harder to combine. Immunology, virology, growing a pathogen, and stabilizing a dangerous mutation all require different skills. The people who could connect those pieces are few and far between. AI is closing those gaps. Models trained on the biological literature can connect ideas across subfields that few researchers have mastered alone. These models can make someone with relevant training in one area more capable and dangerous by helping them learn key steps in another area.
Two reports this year — one from Harvard, one from RAND — examined these issues and reached similar conclusions independently: AI could broaden the range of actors able to mount a large-scale biological attack, while making existing state programs more capable, too. Neither claims the most dangerous thresholds have been crossed though neither rules it out. The safer assumption is that we’re already close enough not to wait and find out.
The State Department’s annual declassified intelligence report has repeatedly concluded that Russia and North Korea have offensive biological weapons programs, while raising concerns about China and Iran. In 2024, The Washington Post documented a major expansion at Sergiev Posad-6, a former Soviet bioweapons site outside Moscow, including new high-containment buildings. Most of the activity that the declassified intelligence reports point to predates AI. With AI, state programs can expand the number of novel designs.
Non-state actors — terrorist groups and others — have also tried to get access to these weapons by building capabilities themselves or attacking facilities that already have the capability. We are talking not about a kid in a basement building the weapon but about serious actors with resources, enabled by biological engineering tools and fueled by AI.
And then, there is the second barrier — which is also eroding quickly.
A paper published this year in Nature Communications described an experiment in which researchers ordered genetic fragments of the 1918 influenza virus and 36 of the 38 DNA synthesis companies sent the sequences. It was enough for a skilled person to reconstruct the virus. While the U.S. Select Agent Program, a federal program overseen by the CDC and th Department of Agriculture, prohibits intact genetic material capable of producing dangerous pathogens, shorter, readily assembled fragments remain unregulated.
But here is the bigger point: Even if we had a perfect screening system that prevented people from ordering DNA fragments that could later be reassembled, it would not be enough. DNA synthesis is a global market, and a sequence blocked in one country can be ordered from another. Someone, somewhere, will get access to the materials they want if they want it badly enough.
The threat landscape is moving faster than our defenses. A genetic blueprint created by AI can become an actual pathogen in weeks. Yes, it is still hard to turn an idea into an actual pathogen — the failure rate is high — but enough actors are trying that some of them will succeed.
Biomedicine is entering one of its most productive periods in decades, driven substantially by these same tools. Broad restrictions would slow research into diseases we could otherwise understand and treat faster. There is no reasonable way to put in restrictions to prevent the use of tools to build bioweapons without thwarting progress in the next generation of therapies.
So what else to do? We need a layered defense, which begins with access and model design. It’s reasonable for verified researchers to have access to leading-edge biological models for legitimate work, but appropriate oversight must be required. And the developers should test models for dangerous capabilities before release.
Governments could require DNA synthesis providers to verify customers, screen for split orders, and undergo independent third-party audits to make sure the system is working. While there is broad consensus that this should be done, whether it will be particularly effective is not clear.
Similarly, getting open-weight AI developers to adopt that same testing standard and guardrails as the frontier AI labs (such as Anthropic and OpenAI) is worth pushing for, but expecting full compliance is unrealistic. And it’s worth remembering that those models, once released, can’t be recalled.
Much of the AI regulation debate is about speed, autonomy, and who controls the frontier. A temporary slowdown might buy time on all three.
But that time is only worth buying if it goes toward building our defenses. We can’t depend on every lab, every country, and every open-source developer getting biosecurity right. They won’t. Over time, that means the development and potential use of biological weapons becomes more likely. Defenses against this threat take years to build. We don’t have years to spare.
Ashish K. Jha, a former White House Covid-19 response coordinator, is a senior fellow at the Belfer Center at Harvard Kennedy School. He is also co-founder and CEO of BioRadar, a public benefit company that builds systems to detect biological threats.