When coding agents write most of a company's code, the next thing to break is not the code review. It is the machine that runs the tests. On September 14, 2026 an Anthropic engineer published what that looked like inside Anthropic: CI jobs up 25 times in six months, and a test-selection service patched three times before it was rebuilt. Each patch bought less time than the last.
This post is for the engineering lead whose CI queue or bill has grown since the team adopted coding agents. It explains the Anthropic post for a reader who has never run test impact analysis, then turns its lesson into a decision for a team of ten rather than a lab. Every figure is Anthropic's, about Anthropic's own codebase. None of them is a benchmark of anyone else.
- 01The growth was compound, not linear.8x more code per engineer, 10x more tests and a nominal number of new engineers produced 25x the CI jobs in six months, by Anthropic's account.
- 02Each patch bought less than the one before.Doubling cores lasted 70 days. Sharding lasted 29. Daily restarts lasted under a day. Anthropic's own description is a bumpy path.
- 03The fix was to take state out of the process.A stateless set of workers writing to an in-memory journal replaced a single process holding every test's history. One engineer, three weeks.
- 04The advice is a planning number.Anthropic tells teams to assume their architecture will be at 25x load within two quarters, whether they build or buy.
01 — FiguresThe numbers Anthropic published #
The post opens with the cause before the effect. Anthropic's engineers ship eight times as much code per quarter as they did in the 2021 to 2025 period, and Claude writes 80 percent of it. Claude also does much of the review. Once writing and reviewing both speed up, the author says, the pressure moves to continuous integration, the system that runs tests on every proposed change.
Per engineer, per quarter
Compared with the 2021–2025 baseline. Anthropic says Claude authors 80 percent of it.
Growth in tests across the codebase
With, in the author's words, a nominal number of added engineers.
Increase over six months
Not every test runs on every change, which is what the rest of the post is about.
The post gives no cost figures and none are inferred here. It also does not claim the 80 percent applies anywhere else. Treat the three numbers as a description of one large monorepo run by people with unusually heavy agent use, and read the pattern rather than the values.
02 — DefinitionsWhat test impact analysis is #
Most teams run every test on every change. That is simple and, as the post says, works up to a point. Past that point the CI gate gets long, expensive and less trusted, because a failing test that has nothing to do with your change is noise you learn to ignore.
Test impact analysis, also called test selection, is a service that picks which tests to run for a given change. Anthropic's is deterministic: it chooses based on recorded past results and on which packages the change touches. The post notes that vendors sell this as a product, so it is not an exotic idea. What is specific to Anthropic is the load.
The service has two parts that must stay in step. A listener records the result of every test in every CI run. A selector reads that history and decides which tests each open pull request needs. The failure mode is lag. If the listener falls 20 minutes behind, the post says, tens of thousands of test updates have not reached the selector. Then a bad merge fails tests for everyone, a flaky dependency blocks merges, and a newly fixed or added test does not run until the listener catches up.
The original design ran as one process, because keeping a running history per test seemed to need a single writer. That decision is the whole story. A single writer cannot be split across machines, so every fix short of removing it was a delay.
The post makes a point that applies to any team, not just a lab. A human engineer can glance at a red test and know it is not theirs. An agent needs to be told, and given a precise set of valid tests it can verify its own work and iterate. So the more agents a team runs, the more the quality of test selection decides how much of their output is useful. The agents also change the shape of the work: Claude prefers smaller pull requests, and agents push overnight and at weekends, so the floor of CI activity rises while the peaks stay bursty.
03 — The recordThree patches, three shrinking gains #
By October 2025 the service was straining and the team was paged two days running. What followed is the useful part of the post, because it is a dated record of quick fixes with the time each one bought.
| Source: Anthropic engineering post, September 14, 2026. Months and durations are as the post states them. | |||
|---|---|---|---|
| Patch | When | Time bought | Why it ran out |
| --- | --- | --- | --- |
| A bigger machine | October 2025 | 70 days | Doubling cores on a single process delays the ceiling, it does not remove it. The post says the team knew it would be fleeting and that ownership of the service was unclear. |
| Sharding by package | February 2026 | 29 days | The listener needed one writer per package, not one overall, so each package got a shard with its own worker. Growth outran it in a month. |
| Daily restarts | March 2026 | under 1 day | The process hit its memory limit by mid-afternoon on most weekdays. Restarting left it further behind each day, and when it fell more than an hour behind, results went unrecorded. |
The post is careful about what the missed results meant. CI still ran on those pull requests and untested code did not reach production. The selector was simply choosing from stale data, which mostly meant running tests that were already flaky or failing for everyone. That is the quiet cost of lag: not danger, but wasted compute and eroded trust.
One detail is worth keeping. During the sharding period the author ran a long-lived session of an internal version of Claude Tag, Anthropic's Slack product, that watched the service and pinged him whenever the listener fell more than 50,000 jobs behind. The post says Claude often argued for a full overhaul and the humans usually chose another patch. The model was, in the record as published, right earlier than the team.
The point is that each of these techniques bought a fraction of the time they did a year ago. On the other hand, overhauling and completely redesigning a service also takes a fraction of the time and is much more sustainable now that writing code is no longer the bottleneck.Anthropic engineering post, September 14, 2026
04 — The fixThe redesign #
The redesign removed the single writer by giving the service a database, specifically an in-memory data store. Any listener worker can now take any result, append it to a journal in the store, and move on holding nothing. That makes the workers stateless, which means more of them can be added on more machines. A small separate consumer rolls the journal up into per-test history every few seconds, and the selector reads from that.
Anthropic says the distributed version costs more to run but is far easier to scale and to profile than one overloaded process. The chart in the post shows the backlog of unprocessed results, which used to build most days and grow week on week, flat after the cutover. The tuning of journal size and worker count was done, the post says, largely by Claude on its own. The whole project took one engineer three weeks, which the author says would have been nearer a quarter a year earlier.
The general lesson the author draws is about planning, not technique. Buying a bigger machine, parallelising and restarting are, in his words, "not the insights to take from this article". The insight is that each of them now buys a fraction of the time it used to, while a full redesign costs a fraction of what it used to. The ratio between patch and rebuild has moved, and it moved because writing code stopped being the slow part. We made a related argument about measuring agent output in our guide to evidence of completed work: usage numbers rise first, and the systems downstream of the agents feel it before the metrics do.
05 — DecisionsThe decision for a smaller team #
Almost nobody reading this runs a monorepo at Anthropic's scale. The question is what a team of five to fifty engineers, now running agents, should take from a lab's experience. Our reading, drawn from the post's own recommendations, sorts by what your CI is doing today.
The last row points at a control the post does not discuss. Agent platforms expose limits on concurrent runs, and setting them is cheaper than scaling the pipeline they feed. We collected those limits in our parallel-agent resource limit reference. For the pipeline itself, the design principles in our CI/CD pipeline design reference predate agents but still hold, with one number changed: the load you plan for. Anthropic's advice is to assume 25 times the current load within two quarters, and to accept that designing a first version for ten to twenty times the visible scale is no longer over-engineering if the budget allows.
If your team is adopting agents and the pipeline is already the constraint, our AI transformation practice starts engagements with the systems downstream of the agents, because that is where the first bill arrives.
06 — Next stepThe patch-to-rebuild ratio has moved #
Record per-test results this week, and plan CI for 25x
Anthropic's account is one company's, about its own codebase, but the shape is general. Agents multiply code, tests and CI jobs faster than any single-process service can absorb, and each quick fix buys less time than the last. Start collecting the test history a selector will need, keep state out of the process that records it, and set the planning number for your pipeline at the load Anthropic reached rather than the load you see today.