# GitHub Outages Show the Limits of Reactive Scaling

> Source: <https://rahmipruitt.me/content/github-outage-reactive-scaling/>
> Published: 2026-08-21 00:27:43+00:00

On August 17, GitHub had a nearly eight-hour outage that is worth studying if you care about agent infrastructure. The interesting part was not simply that GitHub had "too much traffic." The interesting part was that a hidden concurrency limit was crossed, autoscaling was watching the wrong layer, load balancers saturated, and retry behavior amplified demand during recovery.

GitHub says the outage lasted 7 hours and 47 minutes and disrupted GitHub.com, authentication,
GitHub Actions, APIs, pull requests, issues, and Copilot. The Register's writeup says the
immediate cause involved network saturation on load balancers after an Istio sidecar reached a
concurrency limit, with recovery worsened by a latent retry bug in Visual Studio Code.[1](#fn1)[2](#fn2)

That is the hard part about reactive scaling. It works when the bottleneck is obvious and demand is relatively well-behaved. It struggles when the system crosses a hidden limit and clients react by creating even more demand.

I would bet GitHub already has retries, jitter, circuit breakers, load balancing, autoscaling, and every other modern reliability pattern people tell you to use. This is GitHub, not somebody's weekend Django app running on a potato. The lesson is not that those tools are bad. The lesson is that they may not be enough when demand itself becomes reactive.

For people who care about benchmarking, this is familiar. A machine does not fail cleanly after its stable limit. Once it gets past a certain concurrency point, extra requests do not merely fail. They consume sockets, CPU, memory, database connections, queue slots, logs, and worker time. Past the limit, failed work can start reducing the throughput of successful work.

partial failure → retries → more traffic → more saturation → more errors → more retries

What is interesting to me is that, at lower layers of the internet, we already understand pacing. We do not just flood networks with packets forever and hope for the best. We listen for congestion signals. We speed up. We slow down.

But at Layer 7, a load balancer can watch a fleet get half-taken out by capacity pressure and continue routing traffic until the fleet is fully saturated. The standard response is to add more capacity and push the rest back onto the client: "Use better retries. Add jitter. Add exponential backoff. Add a circuit breaker."

Those tools are good. They have carried the industry for a long time. But agents threaten the assumptions underneath them.

To be clear, when I say agent-native infrastructure, I do not mean infrastructure controlled by agents. I mean infrastructure built with agents in mind: bursty traffic, recursive workflows, parallel tool calls, retries, long-running jobs, shared downstream limits, and workloads that can move much faster than human clicking speed.

For most of the web, one human action roughly maps to one request, or at least a predictable handful of requests. Agents change that. One user goal can fan out into many tool calls, API calls, browser sessions, database queries, CI jobs, model calls, retries, and follow-up tasks.

That is why agent-native infrastructure is becoming a real category and not just marketing
soup. Other people are starting to point at the same shift: a16z has written about agent
workloads creating coordination problems around routing, locking, state management, and policy
enforcement across massive parallel execution. 3 I'd add pacing and
fairness to that list — deciding how fast work should be allowed to flow, and making sure one
workload's burst doesn't starve everyone else's.

GitHub is also in a unique position. It has a massive free product that agents, editors, CI systems, and developer tools can all push against at their own pace. Unlike platforms that can simply price every call or hide behind strict quotas, GitHub has to provide a globally reliable developer platform that many machines treat as shared infrastructure.

New architectures like Cursor's Origin are interesting here too. Origin is built on Continuity,
Cursor's storage layer, which rethinks Git hosting around a write-ahead log in S3-compatible
object storage, with replicas catching up from that source of truth. That may improve the
origin side of Git hosting.[4](#fn4)

But even better origins do not remove the downstream problem. If a push, webhook, CI trigger, deployment pipeline, or agent workflow fans out to systems like Depot, Vercel, Buildkite, GitHub Actions, caches, databases, or internal APIs, those consumers still have to absorb the burst. Storage architecture can move a bottleneck. It does not magically pace every downstream system that reacts to the event.

Reactive systems scale supply. Retry storms amplify demand.

When demand reacts faster than supply can scale, the missing layer is not just load balancing.
It is demand shaping.

I built Aquifer around that missing control loop.

Aquifer is a self-hosted agent-native load balancer and traffic coordination layer for agent
workloads. It absorbs bursts into a durable queue, dispatches at a controlled rate, and lets
upstream services dynamically slow it down with `X-Aqueduct-*`

response headers
before overload turns into 429s or cascading failures.[5](#fn5)

Aquifer is not a load balancer that only asks, "Which backend should receive this request?" It also asks, "Should this request run right now?"

It accepts work through an MCP adapter or HTTP API, persists jobs durably, drains them at a controlled rate, sends completion through webhooks, and lets clients stream job status while waiting in line. In the MCP use case, agents call Aquifer instead of racing each other directly against the same backend or external API. Aquifer returns a job ID immediately, dispatches at a controlled rate, and delivers the result to a webhook.

The backend can also send capacity signals back through headers. If the target service is degraded, it can tell Aquifer to slow down. If the service is healthy, Aquifer can increase the rate. The idea is not to destroy the fleet and then ask clients to behave better afterward. The idea is to keep the fleet inside a survivable operating range.

Fairness matters too. Aquifer can isolate queues by user or queue key, so one noisy neighbor does not ruin the experience for everyone else. That matters more in an agent world because one person's workflow might launch a lot more work than a human ever would manually.

I also built EZThrottle Local as a related BEAM-based implementation. It explores the same core
belief from a different runtime: spiky agentic traffic should be paced before it becomes
everyone's outage. EZThrottle Local leans into BEAM tradeoffs like hot-code reloading and
preserving queues during updates.[6](#fn6)

That is a mouthful, so here is the simpler version:

Agent-native load balancing is not about seeing traffic spikes on your backend dashboard. The real pressure often lands on finite shared resources: databases, GPUs, CI runners, caches, search indexes, token services, internal APIs, and third-party platforms. Those resources cannot always scale instantly. Every request beyond their safe limit can start a cascading error cycle.

You may not feel this pain today.

But as agents become more common, the industry's default answer of retries, jitter, circuit breakers, Kubernetes, and reactive autoscaling is going to get challenged. Those tools are still necessary. They are just not the whole control loop.

Load balancers spread traffic.

They do not stop stampedes.
