# Not All Repair Helps: What I Learned Trying to Fix a Failing AI Agent > Source: > Published: 2026-07-30 06:35:49+00:00 Picture a moment every person who runs an AI agent knows. A task is halfway done and starting to go wrong. The agent took a weird turn a few steps back and now it is confidently heading somewhere bad. You have to decide fast on this. Do you step in? And if you do a quick "wait, check your work" nudge will that actually fix it? Or do nothing? Or worse knock a run that was about to recover on its own off the rails? That question is the whole project. Here is the honest short version of what I found. A lot of recent agent research is about **failure attribution** — figuring out *which* step in a long run broke everything. Useful but it stops one step short of what you need when you are on call. Knowing *where* it broke is not the same as knowing *what to do about it*. So I asked a blunter question: **given a failure, which fix actually recovers the run and which ones quietly make it worse?** To answer it without fooling myself I rewind each failing run to the exact step where it went wrong, apply one fix, let it play forward and check the *real* answer against a hard ground truth no LLM grading another LLM. And I always compare against a "do nothing" control, because some runs recover on their own, and I did not want to give my fixes credit for that (or miss a "fix" that's actually worse than leaving the agent alone). First surprise: a decent agent mostly *doesn't* fail in the dramatic ways people worry about. It rarely loops, rarely forgets to answer, rarely fumbles a tool that throws an error in its face. It fails in two quieter ways and both are the same underlying mistake: acting on the *surface* of the situation instead of the real thing underneath. **It makes up an answer it could have looked up.** The fact it needs is sitting right there behind a tool call it just never makes, so it fills the gap with something plausible. Reads "manager: #202," never looks up who #202 is, asserts a name anyway. **It trusts a tool that didn't complain.** Give it a date like `20260730` or a time like `1430` and it hands that straight to a calculator as if it were a number. The calculator doesn't error — it returns *a* number and that non-error result becomes a stamp of approval on a calculation that never made sense. The tool "validated" nonsense. That second one is sneaky because it slips through how we usually categorize errors. Nothing errored. A real tool really ran. Any monitor watching for error signals looks right past it. Here is the part that changed how I think about fixing agents. I tested a bunch of repair nudges. Some phrased as commands some as statements. I expected the *phrasing* to matter. It didn't. What mattered was narrower: **does the nudge give the agent permission to go back and do the missing work?** Look at these two. Grammatically almost identical. The only difference is one short clause telling the agent it is allowed to check. | Repair nudge | Recovery | |---|---| | "The answer must be the manager's name." | 0.16 | | "The answer must be the manager's name — use tools to verify." | 1.00 | Same sentence, one clause of difference. That is the gap between the agent shrugging and re-asserting its made-up answer, versus actually going back, making the lookup it skipped, and getting it right. I call it **action-licensing**: a fix works when it licenses the corrective action not when it is worded a certain way. One honest note, because it is the spirit of the whole thing: I originally thought a different phrasing style was the cause. More data killed that idea — the "winner" just happened to also be the one granting permission. I retracted the old explanation and kept the real one. Skipped-lookup is recoverable. The "tool validated nonsense" one is not — at least not with words. I threw every text-level fix at it and recovery stayed near the floor. Why is kind of elegant: the thing that corrupted the run was not the prompt, it was a *tool result*. The agent is holding a clean, well-formed, completely wrong number that a tool "confirmed." Asking it to re-read the question doesn't remove that false stamp of approval, because the stamp does not live in the prompt. You can't reach it from there. Useful line to draw: some failures are prompt-shaped and some are not and no amount of prompt-fiddling fixes the ones that are not. **Catch it early.** The deeper into a task the agent gets before you catch the skip, the less any fix buys back. Command-style nudges that recover well early collapse toward zero when you catch the mistake late. **Beware cross-model comparisons.** I tried a second, unrelated model. The make-up-an-answer failure showed up there too, so it's not a one-model quirk. But I *couldn't* test recovery on it — it wasn't reliable enough at calling tools in the first place, and every fix depends on the agent re-issuing a tool call. So "the fix failed" and "the model can't call tools" become impossible to tell apart. If you benchmark agent recovery across models, check each one clears a basic tool-reliability bar first, or you'll measure the wrong thing. Plainly, so I don't oversell it: all the recovery numbers come from one primary model (a second confirmed the *mechanism* but couldn't test recovery). It is a clean, synthetic sandbox — that is what buys the hard ground truth, but it is not a messy real-world workload. And a few of the smaller numbers are directional trends, not precise measurements. This is an early, careful result, not a final word.