Autonomous coding agents look great in demos. They read a ticket, write code, run tests, open a PR. Then you put one in a real repo with real credentials and real users, and it starts doing things nobody asked for.
Most of the incidents I've seen (and caused) trace back to five recurring failure modes. None of them are exotic. All of them are catchable if you build the right checks before you ship, not after something breaks.
The agent doesn't just take instructions from you. It takes instructions from every file it reads, every API response it parses, every commit message it summarizes. If any of that content contains text formatted like an instruction, some models will follow it.
What this looks like in practice:
How to catch it:
This is the failure mode that doesn't throw an error. The agent calls the right tool with subtly wrong arguments, gets a 200 response, and moves on like nothing happened. A file gets written to the wrong path. A test gets marked as skipped instead of run. A database query filters on the wrong column.
Because nothing crashes, these bugs often survive code review — the diff looks reasonable, the agent's summary sounds confident, and the actual behavior only surfaces days later.
Agents plan, act, observe, and re-plan. When the "observe" step doesn't produce a clear success or failure signal, some agents just keep trying — the same fix, slightly reworded, over and over, burning tokens and API quota until something external stops them.
This is the failure mode that shows up as a shocking bill, not a visible bug.
| Signal | Likely cause | First check |
|---|---|---|
| Same file edited repeatedly with tiny diffs | Agent can't tell if the fix worked | Is the test/build feedback actually reaching the agent? |
| Tool call count spikes with no new files changed | Agent is re-reading instead of acting | Check for a missing "done" condition in the prompt |
| Token usage climbs but PR never opens | Planning loop with no exit | Add a max-iteration cutoff |
Agents accumulate scope quietly. You grant read access to a repo to answer questions, then write access to fix a bug, then the ability to run arbitrary shell commands to install a dependency, then a broader API token because narrower ones kept causing "permission denied" errors that slowed things down.
Six weeks later, the agent that was supposed to write documentation can also push to main, hit production databases, and call paid third-party APIs. Nobody decided that on purpose; it happened one convenient exception at a time.
Agents work from a snapshot: a cached file tree, a system prompt written months ago, a memory of "the API returns JSON" from before the API changed to return XML. When that snapshot drifts from reality, the agent keeps confidently acting on outdated assumptions, and the errors it produces often look like unrelated bugs rather than a context problem.
This is especially common with MCP servers that cache resource lists or tool schemas at connection time and never refresh them during a long-running session.
None of these five checks require a fancy eval framework. They're mostly logging, assertions, and a handful of adversarial test cases you run before every prompt or model change. The hard part isn't writing them — it's remembering to write them before an agent has write access to something that matters.
If you're setting this up for an MCP-based agent specifically, it's worth locking down the checklist once rather than re-deriving it for every project: session and auth boundaries, tool-call logging, step budgets, and a minimal working server you can point new agents at as a known-safe starting point. That's exactly what I put together as the AgentKitLab MCP Production Checklist — a short, practical pack covering these failure modes plus a minimal working server template and a set of agent-eval test cases you can adapt, available through QuietDesk Studio on Gumroad. Whether or not you use that pack, the underlying habit is the same: assume your agent will eventually be wrong in one of these five ways, and build the tripwire before it needs one.
Written with AI assistance and reviewed for accuracy.