Anthropic’s benchmark progress has nearly doubled. Key evidence is blacked out.

METR, a nonprofit evaluator, estimated on 5 November that Anthropic's benchmark scores have risen at about 1.9 times their earlier rate since October 2025, with a 95% confidence interval of 1.5 to 2.8, and attributed perhaps 80% of the speedup to AI assisting human researchers rather than models running research alone. Anthropic redacted a majority of the evidence METR found most informative, so the public must rely on METR's judgment; the figure falls to about 1.4 against rival labs and ranges from 1.1 to 2.9 across METR's robustness checks. METR calculated about 1.97 from the unredacted report, close to the doubling threshold in Anthropic's Responsible Scaling Policy, while Anthropic's August report said the rate was below 2.

TL;DR and context - METR, a nonprofit evaluator, estimates Anthropic’s benchmark scores have risen at about 1.9 times their earlier rate since October 2025 95% confidence interval 1.5 to 2.8 . - It attributes most of the speedup, perhaps 80%, to AI assisting human researchers, not to models running research alone. - Anthropic redacted a majority of the evidence METR found most informative, so the public must rely on METR’s judgment. - The figure falls to about 1.4 against rival labs and ranges from 1.1 to 2.9 across METR’s robustness checks. - METR calculates about 1.97 from the unredacted report, close to the Responsible Scaling Policy’s doubling threshold. Anthropic’s August report said below 2. - For European buyers, the practical step is to ask vendors which evaluators saw the model and what they could publish. 15 min read METR estimates that AI assistance drives most of Anthropic’s research acceleration. Anthropic withheld much of the evidence behind that judgment, leaving outsiders unable to check its causes. Primary documents: METR’s assessment of Anthropic’s AI R&D acceleration https://metr.org/blog/2026-11-05-anthropic-ai-rd-acceleration-assessment/ | METR’s review of Anthropic’s August risk report https://metr.org/blog/2026-11-05-rd-section-anthropic-risk-report-august-2026-review/ On 5 November, METR, a nonprofit evaluator, published an assessment https://metr.org/blog/2026-11-05-anthropic-ai-rd-acceleration-assessment/ estimating that Anthropic’s benchmark scores have risen at about 1.9 times their earlier rate since October 2025, with AI assistance probably behind most of the speedup. Anthropic redacted a majority of the evidence METR found most informative, so outsiders cannot check the best pages. For European companies that buy or build on Anthropic’s models, that means planning around a vendor’s pace whose supporting evidence the public cannot fully inspect. The headline invites doubt. Benchmark scores do not tell us how a model will perform in a business, and “AI is speeding up AI” has sat on fundraising slides for years. METR spent eleven weeks inside Anthropic and still calls one of its conclusions a guess. Reuters reported https://arynews.tv/anthropics-models-improving-nearly-twice-as-fast-with-help-from-ai-study-finds that the findings amplified recent warnings that AI progress could outpace efforts to assess its risks. The debate has run on forecasts: executives said recursive self improvement was coming, skeptics said it was marketing. METR’s assessment https://metr.org/blog/2026-11-05-anthropic-ai-rd-acceleration-assessment/ offers unusually detailed access to a developer’s research instead. What METR’s 1.9x benchmark finding measures The number comes from AECI, Anthropic’s version of the Epoch Capabilities Index, a composite of many benchmarks. Before October 2025 the models gained about 15 AECI points a year. Since then it is about 29, a rate of 1.9 times with a 95% confidence interval of 1.5 to 2.8. METR’s wider judgment is that Anthropic is improving on well specified tasks at roughly twice its March 2024 to October 2025 rate, with a plausible range of 1.3 to 3 times. METR’s team of 2.5 full time equivalent staff worked inside Anthropic from 12 August to 29 October 2026. According to The Decoder https://the-decoder.com/metr-finds-anthropic-may-have-crossed-its-own-ai-acceleration-warning-threshold-back-in-july/ , they saw benchmark results from unreleased models, training metrics, employee surveys and internal analyses, and held about 15 interviews with Anthropic researchers. They worked on Anthropic devices and took no money from the company. They did not request or receive query access to the unreleased models; they read the lab notebooks and never poked the models. The 1.9 leans on models you cannot use. Anthropic’s system card for Haiku 5.5 reported 1.7; METR’s figure is higher mainly because it included unreleased internal models. The estimate shrinks when other developers enter the comparison. Fit public frontier models from OpenAI, Google and xAI alongside Anthropic’s and the acceleration drops to about 1.4. METR offers two readings: Anthropic may have sat below the frontier before late 2025, which makes 1.9 too high, or its models may be tuned less tightly to benchmarks than rivals’ models, which makes 1.4 too low. The answer also moves with the method. METR’s robustness checks produced anything from 1.1 to 2.9 times, depending on benchmark subsets, model snapshots and statistical choices. How much of the speedup comes from AI assistance? Is Claude now building Claude? METR does not say that. It says AI is not yet making highly significant, fully autonomous contributions to the most important stages of model training. The speedup comes mainly from AI assisting human researchers, though METR notes “signs of life” that independent contributions could grow soon. Assistance sounds mundane in the interviews. One researcher said experiments that once took a day or two to set up are now close to free. We have covered what Anthropic’s code volume does to review https://industrycontents.com/anthropic-scion-ai-code-review-bottleneck/ . Informal Slack polls in October had staff rating the latest internal model at about 12 times more productive, which METR called “very loose evidence,” unrepresentative and probably overstated. It has reason to be wary: a 2025 METR study found AI tools slowed experienced open source developers on the tasks tested, even though the developers believed they were faster. METR’s subjective estimate attributes most of the current acceleration, perhaps about 80%, to AI use in research. It says it is not sure and found no consensus among Anthropic’s staff. One researcher put AI’s share of productivity gains at 15 to 20% and distrusted colleagues’ self reports. Another named new hires as the biggest factor in recent months. METR is confident about one thing: the speedup did not come from faster growth in training compute. The supporting analysis sits in the redacted material, and a planned check of how Anthropic allocates compute was cancelled midway, with Anthropic saying it wanted to focus on risk assessment first. Anthropic’s AI research acceleration and its safety threshold Anthropic’s Responsible Scaling Policy treats a doubling of the pace of progress from automated research as a trigger for heightened scrutiny. The August risk report said the acceleration was below a factor of two. From the unredacted version, METR calculates about 1.97, and calls it plausible that the early warning threshold was crossed by 15 July models, though most relevant researchers at METR and Anthropic lean toward it not having been. METR says the public report understates both the acceleration and the uncertainty around it. The 1.9 is METR’s. Anthropic’s Haiku 5.5 system card put the figure at 1.7, and its other publications describe heavy AI use without claiming a doubling. In “When AI builds itself” September 2026 , it says engineers ship about 8 times as much code per quarter as they did across 2021 to 2025. An earlier study, “How AI is transforming work at Anthropic” December 2025 , reported a 50% average self reported productivity gain and a 67% rise in merged pull requests per engineer per day, and cautioned that lines of code overstate real productivity. An internal R&D Automation Index found that, as of August 2026, Claude led 26% of measured AI R&D work, over 90% involved at least AI collaboration, and no measured subset was fully autonomous. In one controlled test, Claude Mythos Preview made a small model’s training code roughly 52 times faster in April 2026, up from about 3 times for Claude Opus 4 in May 2025, though Anthropic stresses that measures a defined task and not an end to end training speedup. The company’s redacted August risk report says internal AI R&D is significantly faster with AI assistance, “but not yet by a factor of 2.” Its revised IPO prospectus, as reported in November 2026, says it does not expect to be able to safely let Claude build successive generations of itself “without meaningful human supervision or understanding.” The two accounts agree that humans still steer. They disagree about a threshold, and about how much of the evidence you get to see. Why Anthropic redacted METR’s evidence METR submitted a report of roughly 100 pages. In the public version, the cuts removed forecasts for future models, breakdowns by training stage, algorithmic progress estimates, bottleneck analysis and the names of specific benchmarks, according to a summary on AI TLDR https://ai-tldr.dev/releases/metr-anthropic-ai-rd-acceleration/ . The bottleneck analysis is the material behind METR’s conclusion that it found nothing likely to slow progress over the next twelve months. METR wrote https://metr.org/blog/2026-11-05-anthropic-ai-rd-acceleration-assessment/ that Anthropic “redacted a majority of the evidence we found most informative,” so the public must lean on METR’s judgment more than METR considers ideal. Anthropic says the redactions protect confidential information that could speed up other developers. Sam McCandlish, an Anthropic cofounder, wrote on X https://cybernews.com/ai-news/anthropic-redacted-report/ that the company “redacted a lot” and accepted this makes METR’s work harder to check, while arguing that explaining the causes of the acceleration would raise the very risks METR is concerned about. Daniel Kokotajlo of the AI Futures Project replied that a company that calls human extinction plausible has little business citing commercial sensitivity. METR lists its limits: it relied heavily on two Anthropic staff for data and used Claude in the analysis itself. The measurement chain runs through the subject of the audit at several points. The arrangement has one built in check: for this assessment, Anthropic could redact any nonpublic information, while METR could disclose the redactions and whether its findings depended on them. This was separate from the embedded evaluator programme Anthropic announced in October. The report is that mechanism working: we know the evidence is missing because the evaluator told us. METR is a nonprofit with a hard independence rule. It does not accept funding from frontier AI companies or donations directed by their staff, and it reported raising about $71 million in grants https://metr.org/blog/2026-08-14-funding-update/ in August over six months. The labs supply model access and tokens, which METR describes as separate from payment. Critics say the separation is thinner than it looks, pointing to Good Ventures and Open Philanthropy, major funders across the AI safety world, and to Dustin Moskovitz, associated with those groups, who has long backed Anthropic, as OfficeChai reported https://officechai.com/ai/metrs-independence-questioned-after-x-user-highlights-financial-links-between-company-and-anthropic/ . Defenders note that METR publishes its funding sources and that its president, Chris Painter, has said the group takes no money from frontier labs or their employees. Independence from payment is a separate matter from independence from access, and here Anthropic held the redaction pen. Painter has acknowledged that today’s third party evaluations remain voluntary collaborations with limited oversight power. The debate over recursive self-improvement Within a day, New York Assemblymember Alex Bores asked Congress to ban recursive self improvement, or RSI, “this year if possible,” citing the METR report, OpenAI’s stated goal of an automated AI researcher by March 2028, and what he described as Anthropic employees’ belief that they are on track to do so by 2027. Representative Greg Casar reposted him with two words: “Ban RSI.” Nothing has been banned. In September, Representative Ro Khanna outlined plans for a Human Control Over AI Act https://thenextweb.com/news/human-control-over-ai-act-khanna-self-improving-ai , before the METR report. According to CNBC’s reporting, it would bar models from recursively improving themselves or autonomously modifying their core objectives, containment or shutdown controls until a new federal agency approves them, and it would create a licensing body for frontier models. Khanna acknowledged that the capability the ban targets does not exist at the level he wants prohibited, and argued that governments should act first. Speaker Mike Johnson told CNBC he favours voluntary guardrails, and President Trump has said companies will police themselves. Anthropic’s position muddies things: in June its institute https://www.anthropic.com/institute/recursive-self-improvement said “we are not there yet” and urged the world to keep the option of slowing frontier development open, and Dario Amodei called for slowing down in September. The bill’s weak point is definition. METR’s evidence concerns assistance, with humans directing the work. A statute aimed at autonomous self modification would regulate a capability that has not arrived and miss what is happening, which is AI speeding up human researchers. A ban broad enough to cover assistance could also reach ordinary AI coding tools. What European buyers should do If you buy or build on frontier models in Europe, three things follow. First, pace: if a leading vendor is moving at roughly twice its earlier rate, teams may face more frequent decisions about updating prompts, evaluation suites and workflows. The report supports this only for benchmark progress, not real world capability, so it does not establish a migration timetable. Vendor pricing is already shifting; see our note on Claude Fable 5.2 rates and fast mode https://industrycontents.com/news-desk/claude-fable-5-2-pricing-fast-mode/ . Second, evidence: the EU AI Act already gives providers of general purpose models with systemic risk a set of duties under Article 55 https://ai-act-service-desk.ec.europa.eu/en/ai-act/article-55 . According to the Commission’s AI Act Service Desk, providers must run standardised model evaluation including adversarial testing, assess and mitigate systemic risks, report serious incidents to the AI Office, and ensure adequate cybersecurity. Those duties bind the provider but give buyers no automatic access to the underlying evaluation evidence. Third, contracts: ask vendors which independent evaluators have seen the model, what they were allowed to publish and which categories were redacted. METR’s habit of describing its redactions publicly gives you a template for the question. Our guide to deciding when to let AI act independently https://industrycontents.com/when-to-constrain-ai/ offers a way to set limits once you have the answers. The open question is the link between benchmark scores and useful work. METR itself says it remains unclear how those scores map to real world capability. A September paper https://arxiv.org/pdf/2609.15802 by authors including METR’s Tom Cunningham estimated that feedback loops in AI progress are not currently strong enough to produce self sustaining acceleration, though they appear to be strengthening. METR’s report warns that further gains could produce even faster acceleration over the coming year. What to watch, in order - Coming weeks: Anthropic has said it will publish an analysis of the speed of progress inside the company, including where it stands against its Responsible Scaling Policy. - As soon as November: Anthropic’s possible IPO, per Reuters https://arynews.tv/anthropics-models-improving-nearly-twice-as-fast-with-help-from-ai-study-finds . - Releases after the assessment: Anthropic announced Fable 5.2 and Mythos 5.2 https://www.anthropic.com/claude-fable-and-mythos-5-2 on 9 November. Their release alone does not establish that the measured acceleration translates into customer results. - Within six months: METR says it cannot rule out another sharp jump. We are not ready to call this recursive self improvement. But the acceleration looks real, if smaller or larger than 1.9, and the people closest to it mostly describe researchers with very good tools. The usual framing is a coin flip: either the machines are running away or the whole thing is hype. The better question is practical. If the pace is real, who gets to check it, and what is a vendor willing to let them publish? Sources and limits This analysis draws on METR’s public assessment and its review of Anthropic’s August risk report, secondary coverage, Reuters, CNBC reporting on the Khanna bill, the Commission’s AI Act Service Desk and statements by the people named. Figures from secondary coverage are attributed to it. Industry Contents did not contact Anthropic or METR, did not see the unredacted report and could not independently verify any benchmark figure. Redaction details rest on an aggregator summary. Corrections and right of reply are welcome at editorial@industrycontents.com. FAQ Did METR find that Claude is building itself? No. METR says AI is not yet making highly significant, fully autonomous contributions to the most important stages of model training. The speedup comes mainly from AI assisting human researchers. Why did Anthropic redact the evidence? Anthropic says the redactions protect confidential information that could speed up other developers. Cofounder Sam McCandlish also argued that explaining the causes would raise the risks METR is concerned about. Critics say that weakens outside checking. Does a 1.9 times speedup mean Anthropic’s models are twice as capable? No. The figure measures the pace of benchmark progress on tasks with checkable answers. METR says it remains unclear how those scores map to real world capability. Has Anthropic crossed its own scaling policy threshold? Unclear. The August report said acceleration was below a factor of two, but METR calculates about 1.97 from the unredacted version. It calls a crossing by 15 July models plausible, though most relevant researchers at METR and Anthropic lean toward no. Spotted an error? Contact the desk https://industrycontents.com/contact/ . Our corrections policy https://industrycontents.com/how-we-report/ explains how we handle changes. A server reboot, a $10,000 complaint and an AI agent with ideas of its own https://industrycontents.com/codex-server-reboot-ai-agent-permissions/ shows why permissions for AI agents need limits.