Anthropic says a key AI safety alarm has been triggered, days before its IPO

Anthropic disclosed in its amended IPO prospectus on Tuesday that it has met a safety threshold in its Responsible Scaling Policy — the trigger for when AI roughly doubles the pace of the company's model improvement — based on "what we expect from future models and our overall rate of progress, rather than on what we have observed from current models." The same filing set a price range of $70 to $78 per share for an IPO valuing Anthropic at as much as about $1.4 trillion on a fully diluted basis, with trading possibly starting as soon as Nov. 19; Nvidia and Google are each buying $10 billion of stock at the IPO price as part of a $26 billion private placement that also includes AMD, Broadcom, Micron, Microsoft and Taiwan Semiconductor. Independent nonprofit METR estimated last week that Anthropic's models have been improving nearly twice as fast since October 2025, and called Anthropic's August argument that it remained below the threshold "overstated.

A week ago, Anthropic told would-be investors that AI speeding up its own research might "imminently" trigger one of the company's key safety alarms. On Tuesday, in the same filing https://www.sec.gov/Archives/edgar/data/2130357/000162828026072484/anthropicpbc-sx1ax2.htm that set a price range for its record-setting IPO https://finance.yahoo.com/markets/article/anthropic-seeks-14-trillion-valuation-in-ipo-130300928.html , it said that has now happened. "We made a determination that we have met this threshold," the company wrote in its amended prospectus https://www.investopedia.com/terms/s/sec-form-s-1.asp . It based the call on "what we expect from future models and our overall rate of progress, rather than on what we have observed from current models." The safety threshold sits in Anthropic's Responsible Scaling Policy https://www.anthropic.com/responsible-scaling-policy — the company rulebook for adding tougher safety measures as its AI gets more powerful. This one is triggered when AI is doing enough of Anthropic's research to roughly double how fast its models improve. The policy says that pace would pack two years' worth of AI progress, at its 2018-2024 rate, into a single year. The filing says AI "has already been accelerating our research and development." Crossing the line means more safeguards, not a shutdown. The fear is AI will improve itself faster than people can check it. In 2023, Anthropic CEO Dario Amodei joined other AI leaders in warning https://uk.finance.yahoo.com/news/ai-leaders-warn-risk-extinction-171046396.html that the "risk of extinction from AI" should be a global priority alongside pandemics and nuclear war. Anthropic says humans stay in charge. It doesn't expect to safely use Claude "to fully autonomously design and build successive generations of frontier models without meaningful human supervision or understanding," the filing says, and it is weighing policy changes "to keep humans in control." An outside group sees the same speedup. METR, an independent nonprofit that tests AI systems, estimated last week https://metr.org/blog/2026-11-05-anthropic-ai-rd-acceleration-assessment that Anthropic's models have been improving nearly twice as fast since October 2025. In a separate review https://metr.org/blog/2026-11-05-rd-section-anthropic-risk-report-august-2026-review , METR called Anthropic's August argument that it was still below the threshold "overstated" and said it was "plausible" the line was crossed by mid-July. A $1.4 trillion debut Anthropic set its price range at $70 to $78 per share on Tuesday as it began its roadshow https://www.investopedia.com/terms/r/roadshow.asp , the round of investor meetings before pricing. That values it at as much as about $1.4 trillion, counting every share that could exist https://www.investopedia.com/terms/f/fullydilutedshares.asp . Trading could start as soon as Nov. 19, according to reports https://finance.yahoo.com/markets/article/anthropic-seeks-14-trillion-valuation-in-ipo-130300928.html . Read more: Everything you need to know ahead of Anthropic's IPO Some of the market's biggest names are tied up in the IPO. Nvidia NVDA https://finance.yahoo.com/quote/NVDA/ and Alphabet's GOOG https://finance.yahoo.com/quote/GOOG/ , GOOGL https://finance.yahoo.com/quote/GOOGL/ Google are each buying $10 billion of Anthropic stock at the IPO price, part of a $26 billion private placement with AMD AMD https://finance.yahoo.com/quote/AMD/ , Broadcom AVGO https://finance.yahoo.com/quote/AVGO/ , Micron MU https://finance.yahoo.com/quote/MU/ , Microsoft MSFT https://finance.yahoo.com/quote/MSFT/ , and Taiwan Semiconductor TSM https://finance.yahoo.com/quote/TSM/ . Amazon AMZN https://finance.yahoo.com/quote/AMZN/ and Google already own big stakes.