Tests grew 10x and CI jobs 25x in six months, forcing a three-week rebuild of Anthropic's test-selection system.
By [Ryan Merket](https://runtimewire.com/author/ryan-merket)
· Published
Primary source: [X - Addy Osmani](https://x.com/addyosmani/status/2099577600159158765)
Why it matters #
Anthropic's self-reported gains expose the next cost of AI coding: code arrives faster, while testing, review and deployment systems must absorb machine-scale volume.
Anthropic says Claude now authors about 80% of the code merged into its repositories, while its engineers ship eight times as much code per quarter as they did during the 2021-to-2025 period. The output has shifted the engineering constraint downstream: Anthropic's test count grew 10x and its continuous-integration job volume climbed 25x in six months.
Addy Osmani (@addyosmani), the former Chrome developer-experience leader whose move to Anthropic RuntimeWire reported last week, surfaced the figures Monday. He clarified that the 80% figure measures lines merged into production that Anthropic can clearly attribute to Claude. The remaining 20% combines human-written code and other artifacts.
The figures come from a September 14th engineering post by Sachin Malhotra, an engineer responsible for Anthropic's CI infrastructure. They are company-reported operational metrics rather than an independently audited productivity study. The eightfold comparison also measures code volume, which does not establish an equivalent increase in features, revenue or software quality.
Anthropic's experience still provides a detailed account of what happens when code generation and review begin moving at agent speed. Writing code stopped being the main constraint. Testing infrastructure became the choke point.
Three patches, each shorter-lived
Anthropic uses a deterministic test-impact analysis service to choose which tests should run against each proposed change. A listener records results from CI jobs, while a selector uses that history and package relevance to assemble the tests for a new pull request. Running every test against every change would be slower and more expensive.
The original service kept its test history inside a single process. As CI traffic climbed, that listener began falling behind. Malhotra wrote that 20 minutes of lag could leave tens of thousands of test updates unapplied, causing the selector to work from stale information. The likely result was extra investigations, flaky tests blocking merges and newly added or repaired tests failing to run promptly. Anthropic said this did not mean CI was skipped or untested code entered production.
The warning signs appeared by October 2025. Anthropic first doubled the processor cores assigned to the service, a change that held for 70 days. When the backlog returned in February, Claude generated code that split package state across separate workers. That patch lasted 29 days.
By March, the process was hitting its memory ceiling during the workday. Daily restarts bought less than a day and gradually left the service further behind. Anthropic had reached the point where restarting the singleton was creating another operational problem rather than clearing the original one.
Claude helped rebuild the system it overwhelmed
Anthropic replaced the stateful singleton with a distributed design backed by an in-memory data store. Listener workers can now process any result, append it to a journal and continue without retaining local state. A separate consumer periodically converts that journal into per-test history for the selector.
Malhotra said one engineer completed the redesign in three weeks, compared with an estimated quarter under the previous development process. Claude also handled much of the subsequent tuning, including adjustments to the journal and worker count. Anthropic says the service has remained stable since the cutover, although the distributed architecture costs more to operate.
That trade-off sits underneath the headline productivity figures. Anthropic can produce code faster because it is also spending more on the systems that test, review and govern that code. The company's July account of its AI-heavy development process described multiple automated reviewers, risk tiers, human approval gates and continuous security checks surrounding Claude-generated changes.
The 25x growth in CI traffic came from several sources. Anthropic says Claude favors smaller, more granular pull requests, increasing the number of test cycles. Agents also work overnight and on weekends, raising the baseline load even while human engineers continue to direct and approve a significant share of changes.
Malhotra's recommendation is blunt: teams deploying coding agents should design early infrastructure for loads 10 to 20 times larger than the apparent starting requirement, budget permitting, and assume CI could face 25x traffic within two quarters. Anthropic arrived at that position after three temporary fixes lasted 70 days, 29 days and less than one day. Agentic coding made the permanent rebuild faster. It also made every delay in starting it more expensive.