{"slug": "ai-versus-throughput", "title": "AI versus Throughput", "summary": "GitHub attributed its recent poor availability to an \"acceleration in agentic development workflow,\" according to a company blog post, illustrating how AI-driven development can shift a software organization's bottleneck rather than raise its overall throughput. Applying the Theory of Constraints, the analysis argues that speeding up coding only builds unfinished inventory — spreadsheets, Jira tickets, Git commits, and undeployed pull requests — while validation and verification stages such as pull-request review, builds, tests, and security scans become the new constraint. The piece warns that a microservice architecture with frequent deployments can feel 88% available instead of 99%, since a user request fails if any service in its call tree breaks.", "body_md": "According to the Theory of Constraints (ToC), every system has exactly one bottleneck that determines the throughput of the overall system. ToC originated with manufacturing, but its results have been observed to be quite general.\n\nIn the diagram above, the whole process cannot deliver any faster than Station 2 is able to transform its buffer stock of unfinished inventory into its own output. Suppose you make Station 3 run ten times faster than it did last week, leaving everything else the same. The overall process throughput won’t change. All that will happen is for Station 3 to be starved of inputs and forced to idle part of its time.\n\nSuppose instead you invest in the throughput of Station 1, increasing it by ten times while again leaving everything else unchanged. The overall process throughput *still* won’t change. Station 1 will just overproduce and build up inventory of unfinished work waiting for Station 2. In some ways this is even worse, because inventory has a carry cost. Not only that, but the bigger the piles of unfinished inventory, the longer it takes something *new* to get through the whole process, since the new piece has to wait in those queues of unfinished work.\n\nThe only place you want to invest in capacity improvement is at the bottleneck. Increase capacity at Station 2 by ten times and you should get ten times the throughput. Why do I say you “should get ten times the throughput” instead of “you *will* get ten times the throughput?” Two things can go wrong. Station 1 might not be able to produce enough to keep Station 2 busy at its new higher capacity. That would indicate Station 1 is now the new bottleneck. Or Station 3 might not be able to consume enough of Station 2’s outputs, causing an inventory buildup. That would mean Station 3 is the new bottleneck.\n\n## Manufacturing Software\n\nWriting code is not making car parts. Even so, it has proven useful to look at the overall *system of work* in terms of its throughput, cycle time, and failure rate. In an organization that creates software, the inventory looks like spreadsheets, Jira tickets, Git commits, and undeployed pull requests. The work stations are product managers, designers, developers, merge queues, build pipelines, and pre-prod environments.\n\n“Throughput” here means how many features you deliver in a given time frame. If most of your features are valued-producing then higher throughput means higher profits.\n\nSo what happens when you adopt AI driven development? The short version is that you’ve increased the capacity of Station 2. Does that mean you’ve increased the overall throughput? Maybe, maybe not. You might just be overproducing inventory.\n\n[Github](https://github.com) has recently [blamed their poor availability](https://github.blog/news-insights/company-news/an-update-on-github-availability/) on “acceleration in agentic development workflow”. That sounds like a machine breaking.\n\nYour build pipeline may be breaking too. Is your CI machinery keeping up with the increased PR volume? Reviewing pull requests, running builds, tests, security scans… these all consume the capacity in the later stages of your delivery process. Watch out for increasing queue depth there! That’s unfinished inventory building up and it would mean the bottleneck is not coding but validation and verification (V&V).\n\nYou might also see an increased number of deployments into your production environment. This is good, mostly. If your services are configured to hard kill on deployments then every deployment means failed requests for some number of users. Maybe that’s tolerable at a low deployment frequency. But keep in mind that a user request fails if *any* of the services along its call tree break. A microservice soup frothing with deployments will feel 88% available instead of 99%.\n\n## Double Bind\n\nIt’s a cruel irony: our best way to catch AI mistakes is to increase V&V work, but that slows down the new bottleneck stage. We need to increase the capacity of build systems to match.\n\n- If you depend on external vendors during build and deploy, you need to monitor them carefully to see when its time to pull those dependencies in house.\n- Platform teams are usually understaffed and their tools undersized. Keep an eye on your artifact and container repositories.\n- SAST and DAST capacity are going to be very tough to increase since their output still often requires human review.\n- We typically instruct AI agents to write unit tests but we don’t tell them to measure test execution time or consolidate tests. As a result, test times have a way of creeping upward. Consider replacing comprehensive unit tests with acceptance tests.\n\n## Converging Parallel Workstreams\n\nSo far I’ve mostly talked about a single flow through a single process. In reality, each team is like their own production line. Those lines all converge at a big integration step. Defects creep in at the interfaces between services, so integration is where many defects are uncovered. Great! We *want* our build process to catch those defects and reject changes that would break the system. However, each rejected build means work goes back upstream. Coding is not the bottleneck any more, so we shouldn’t worry about adding to the load on the coding agents. However, once an agent fixes a defect it pushes another change forward into V&V. This added load is called “failure demand.”\n\nAnother lesson from ToC is to do more work upstream to avoid putting defective pieces into the bottleneck. Does that create a paradox? How do I avoid putting defective work into the stage that detects the defects?\n\nOne way is through more cross-service quality checking during code creation. Local builds run by agents should catch more of the integration defects. Perhaps via property-based testing. Maybe we move to strongly typed API specifications outside of the service code, taking a lesson from the days of [IDLs](https://en.wikipedia.org/wiki/Interface_description_language).\n\n## Summing Up\n\nOur architecture and processes evolved for human-driven coding rates. Code production is no longer the bottleneck, if it ever was. Right now, it looks like the next bottleneck will be version control, build, test, and deployment systems. The lag time between “code commit” and “running in production” has gotten longer and longer as we’ve added more automated checks… and we need to add even more to catch agentic errors. Our build systems aren’t prepared for the volume of work. We need to elevate the capacity of those processes and simultaneously offload work to upstream stages where possible. It’s OK to generate more code if it reduces failure demand or validation time in the build pipeline.", "url": "https://wpnews.pro/news/ai-versus-throughput", "canonical_source": "https://www.michaelnygard.com/blog/2026/05/ai-versus-throughput/", "published_at": "2026-09-14 03:05:54+00:00", "updated_at": "2026-09-14 03:27:48.491518+00:00", "lang": "en", "topics": ["ai-agents", "developer-tools", "ai-tools", "mlops"], "entities": ["GitHub", "Theory of Constraints", "Jira", "Git"], "alternates": {"html": "https://wpnews.pro/news/ai-versus-throughput", "markdown": "https://wpnews.pro/news/ai-versus-throughput.md", "text": "https://wpnews.pro/news/ai-versus-throughput.txt", "jsonld": "https://wpnews.pro/news/ai-versus-throughput.jsonld"}}