The Claude maker wants mandatory third-party audits and federal power to halt dangerous AI deployments, even as it fights the Pentagon over safety guardrails.
Anthropic CEO Dario Amodei just asked the federal government to do something most tech executives would rather chew glass than suggest: regulate his own industry. Hard.
On June 10, Amodei unveiled what he’s calling an “Advanced AI Framework,” a regulatory blueprint that would give the US government the authority to block AI models deemed to present “unacceptable risks.” That includes models capable of enabling cybersecurity threats or aiding in the development of biological weapons.
What the framework actually proposes #
The core of Amodei’s proposal centers on mandatory third-party testing for frontier AI models. Before a powerful new model hits the market, independent auditors would stress-test it for dangerous capabilities, similar to how pharmaceuticals go through clinical trials or aircraft undergo safety certifications before carrying passengers.
If a model fails those tests, the government could block its deployment entirely. The framework also proposes significant civil penalties tied to a company’s global annual revenue for noncompliance.
Amodei is also advocating for a graduated approach, starting with softer regulatory measures that could intensify over time as AI capabilities advance.
There’s also an “Economic Policy Framework” baked into the proposal, aimed at supporting workers displaced by AI-related job losses.
The Pentagon problem #
In February, Anthropic refused Pentagon demands to remove safety guardrails from the Claude family of models. Those guardrails are specifically designed to prevent the use of AI in autonomous weapons systems and domestic mass surveillance. The refusal sparked a legal dispute centered on supply-chain risks.
Just days before Amodei’s announcement, there were calls within the broader AI industry for a in advanced AI development, reflecting growing concern about the pace of progress outstripping anyone’s ability to manage it.
What this means for the AI investment landscape #
The framework could create a meaningful competitive moat for companies that already invest heavily in safety infrastructure. If third-party audits become mandatory, firms that have been treating safety as an afterthought will face steep adjustment costs. Companies like Anthropic, which have built safety considerations into their development process from day one, would have a structural advantage.
Revenue-based penalties add another layer of risk for investors to consider. A company generating tens of billions in annual revenue could face penalties in the hundreds of millions or more for deploying a model that regulators flag as dangerous.
The Pentagon dispute also raises questions about the government contracting pipeline for AI companies. Anthropic’s refusal to strip safety features suggests that some companies may be willing to walk away from government dollars rather than compromise on principles.
Disclosure: This article was edited by Editorial Team. For more information on how we create and review content, see our