AI Will Be Wrong Sometimes. What Then?

A developer building an LLM-powered customer support agent has documented four deterministic failure-mitigation patterns that avoid asking the model to be correct, including score-threshold filtering to suppress weak search matches, circuit breakers for unreachable carrier APIs, and a deterministic keyword-classifier fallback that refuses rather than guesses. The project, published with a companion repository and a failure-modes document mapping each mitigation to a test, argues that AI systems rely on the same circuit breakers and range checks used in ordinary microservices, with the only novelty being which component is unreliable.

Four ways this system goes wrong, and the code that catches each one Part 11 findings of an experiment: building an LLM-powered support agent with deterministic boundaries. The companion repo https://github.com/antoniolopescorreia/reliable-ai-support contains the full code. The agent asks to refund order ORD-999. There is no ORD-999. There never was. Nothing in this codebase checks for hallucinations. The request dies anyway, in the same line of code that stops a customer reading someone else's order — a lookup that takes the authenticated session and finds nothing. An invented id and a stranger's id are the same thing to a query with a WHERE clause on the customer. That's the pattern for every failure mode in this post. None of the mitigations ask the model to be right. Search always returns something . Ask "who won the game last night?" and the ranker dutifully surfaces the rate-limits article, because both contain the word "the". The honesty has to live in the score, not the ranking. Below a threshold, the best match is treated as noise and the agent declines: php Optional<KnowledgeArticle best = retrieved.stream .filter scored - scored.score = MIN ANSWER SCORE .map ScoredArticle::article .findFirst ; "I don't have an answer for that" is a real answer. A confidently wrong one costs more than a shrug. The carrier's tracking API is the one dependency I don't control, so it will be unreachable at some point. When it is, the agent must not throw, and must not improvise. public <T T callOrFallback Supplier<T call, Supplier<T fallback { if isOpen { return fallback.get ; } try { T result = call.get ; consecutiveFailures = 0; return result; } catch RuntimeException failure { recordFailure ; return fallback.get ; } } Two failures open the breaker, a cooldown closes it, and the customer gets a sentence: "I can't reach the carrier right now, so I can't confirm where this order is. Nothing has changed about the delivery itself." The detail I'd have got wrong a few years ago is keeping that separate from the denial message. "Carrier unavailable" and "not your order" are different answers, and collapsing them teaches customers that unavailable sometimes means not yours . One test asserts exactly that. Same shape, different dependency. FallbackIntentClassifier catches the provider failure and hands the message to the deterministic keyword classifier. Degrading like this is only safe because of what sits underneath. The fallback understands fewer messages, and everything it can't understand is refused rather than guessed at. A fallback that guessed would be worse than an outage. php flowchart LR M "Customer message" -- C{"Classify"} C -- |"provider down"| FB "Deterministic fallback" C -- U{"Understood?"} FB -- U U -- |"no"| R1 "Refused" U -- |"yes"| L{"Scoped lookup"} L -- |"invented or<br/ not yours"| R2 "Refused" L -- |"found"| K{"Carrier call"} K -- |"unreachable"| F "Fallback sentence" K -- |"ok"| A "Answer or proposal" classDef step fill: eef2f6,stroke: 8fa3b8,color: 24313f classDef decision fill: f7f4ec,stroke: b3a988,color: 24313f classDef bad fill: f5ecec,stroke: c4a29e,color: 5a4442 classDef good fill: ecf2ed,stroke: 93b39d,color: 3d5344 class M,FB,F step class C,U,L,K decision class R1,R2 bad class A good Read the mitigations together and there are only four: None of these are AI ideas. They're the circuit breakers in any microservice mesh, the range checks on any industrial sensor, the "sanity failed, hold position" branch in any control loop. The novelty in an AI system is only which component is unreliable — not what you do about it. Every row is in docs/failure-modes.md with the test that proves it. A row with no test is a row with no mitigation, and I'd rather the table say "planned" than imply coverage I don't have. The four rows here are the failures I could think of. That's exactly the wrong sample. The Planned half of that table is more honest about where this is fragile: approval fatigue, double execution after a retry, and a knowledge base that drifts while every test stays green. The first is a human problem, the second is an idempotency problem, and the third is invisible by construction — nothing turns red when answers quietly get worse. Which failure mode in your system is real, known, and still described as "we should handle that at some point"?