Originally published on Loop & Retry — field notes on building LLM agents that survive production.
The demo works. It always works — that's what a demo is. You gave the agent a representative task, it took the happy path, and it produced the right answer in front of an audience. What the demo proved is that the agent can succeed once, on an input you chose. What it did not tell you is the number you actually need: how often it will fail on the inputs you didn't choose, under load you didn't apply, from users who don't know or care what shape you expected. That gap — between "succeeds once on a good input" and "fails 8% of the time on the real distribution" — is where production failure lives, and most of it is predictable before release if you test the right things.
This is the pre-release companion to measuring agent failure in production. That post is about instrumenting failures once they're live. This one is about forecasting them while you can still cheaply do something — because the cheapest failure to fix is the one you caught in a test harness, and the most expensive is the one your users find. The point isn't to prove the agent works. You know it can. The point is to make it fail on purpose, on your schedule, and read the rate.
The demo and the deployment run the same code. Four things differ, and each one is a failure source the demo structurally can't show you:
Every one of these is testable before release. The reason they usually aren't is that testing them requires deliberately trying to break the thing you just got working, which is psychologically the opposite of what a green demo makes you want to do.
Here's what I run before shipping, ordered by how much production failure each one predicts per hour of effort.
1. Failure injection: force the errors and measure recovery. Don't wait for a tool to time out in production to learn what the agent does. Inject the failure in a harness — make the tool return a 500, a 429, malformed JSON, an empty result, a timeout — and assert on the recovery, not just the happy path.
FAULTS = {
"timeout": lambda: (_ for _ in ()).throw(TimeoutError()),
"http_500": lambda: {"status": 500, "body": "internal error"},
"http_429": lambda: {"status": 429, "retry_after": 30},
"malformed": lambda: {"status": 200, "body": "{not valid json"},
"empty": lambda: {"status": 200, "body": "[]"},
"wrong_schema": lambda: {"status": 200, "body": '{"unexpected": true}'},
}
def probe(agent, fault_name):
"""Run the agent with one fault injected; record what it does."""
result = agent.run(task=REPRESENTATIVE_TASK, tool_fault=FAULTS[fault_name])
return {
"fault": fault_name,
"recovered": result.completed and result.correct,
"retries": result.retry_count, # did it retry a non-retryable error?
"cost": result.total_cost, # did one fault blow the budget?
"escalated": result.asked_for_help, # did it fail loudly or silently?
}
for fault in FAULTS:
print(probe(agent, fault))
The output is a failure-mode matrix: for each fault, did the agent recover, and how much did failing cost? The two rows that matter most are malformed
/wrong_schema
(does a poisoned tool result cascade, per how failures cascade?) and http_429
/http_500
(does it retry a permanent error into a runaway bill?). If the agent retries malformed
fifty times or escalates nothing when it gives up, you've just found a production incident in a unit test.
2. Distribution testing: run the real tails, not the mean. Pull a sample of actual historical inputs — or the closest proxy you have — and run the agent across all of them, not the three you'd demo. Sort the results by failure and cost. You're looking for the shape of the tail: what fraction fail, and are the failures concentrated in an input class you can characterize (long documents, a specific language, a field that's often empty)? A characterizable failing class is a fix; a diffuse one is a redesign. Either way the rate on a representative sample is your single best point estimate of the production failure rate, and it's available before you ship.
3. Load testing with the cost model attached. Run the agent at production concurrency against a staging backend and watch two things: the failure rate under contention (does it climb when workers share rate limits?) and the cost per task under retry (does the retry multiplier you budgeted for hold when failures cluster?). Concurrency-induced failures are the ones that make people say "it worked in staging" — because staging ran it once. Run it a thousand times in parallel and the throttling, the pool exhaustion, and the correlated retries all show up.
4. Canary and shadow before full traffic. The honest pre-release test is a small slice of real traffic. Shadow-run the agent against production inputs without acting on the outputs, and compare its results to the incumbent (a human, an old system, a simpler agent). This catches the distribution and load problems the harness approximated, on the genuine article, with the failures contained because nothing downstream consumes the shadow output yet. A canary that fails at 8% when you expected 1% is a launch you're glad you staged.
Worth naming, because they absorb effort that could go to the four above:
The through-line: predictive tests are the ones that introduce something the demo lacked — a fault, a tail input, concurrency, real traffic. Tests that stay on the happy path, however many you run, just re-prove the demo.
The demo proves the agent can succeed. Predicting production failure is the opposite exercise — making it fail deliberately, at low stakes, and reading the rate before your users read it for you. The failures are coming either way. The only choice is whether you meet them in a test harness or on the invoice.
The injection harness above is a sketch of the pattern, not a framework — the recovery assertions and fault set are where the real work is, and they're specific to your tools. The categories (input distribution, load, adversarial, drift) transfer across providers and models.