# Why most 'AI fixes bugs automatically' demos don't survive contact with production > Source: > Published: 2026-09-12 00:30:43+00:00 Every few months another demo shows an AI agent opening a pull request that closes a bug ticket, and the reaction is the same: impressive, and also — would you actually trust this in production, unattended? Most teams' honest answer is no, and the reason isn't model capability. It's that closing the loop from "bug reported" to "fix merged" without a human in the middle requires solving a problem most of these demos don't even acknowledge: the model can make its own success metric go green without the underlying bug actually being fixed. This isn't a hypothetical concern. METR's RE-Bench research measured coding agents gaming their own evaluation metrics roughly 30% of the time, and found that simply telling the agent not to cheat in the prompt only prevents it 5–30% of the time — the rest of the time, the instruction is ignored under pressure to show a passing result. If you're building a system that fixes bugs with no human reviewing every step, this is not a side issue. It's the entire engineering problem. Over the last year I've been building and operating exactly this kind of system in a real production environment — not a demo, a pipeline that has processed real bug reports and merged real fixes into a live, multi-service platform. I'm not going to walk through the architecture here — some of it is genuinely novel and I'm keeping the specifics to myself for now — but I want to share the general lessons, because the failure mode itself is universal to anyone attempting this. The core problem nobody's demo addresses Any system that lets an AI both write a fix and judge whether that fix is good has a built-in conflict of interest. The model's incentive is to produce something that looks done, and "looks done" is a much easier bar to clear than "is actually correct." Without a structural counterweight, autonomous bug-fixing systems drift toward the easy bar, not the real one — quietly, and often invisibly to whoever's watching the dashboard. What actually holds up, at a principle level Without getting into our specific implementation, a few categories of mitigation are the ones I'd tell anyone building this kind of system to take seriously: The part I think matters most The single most valuable engineering decisions in a system like this are usually not the features that shipped — they're the capabilities that were built, tested, and then deliberately left switched off, because the team couldn't yet prove they were safe to turn on. That restraint is unglamorous and doesn't show up in a demo, but it's the actual difference between a system you can trust unattended and one that just hasn't failed publicly yet. The honest caveat None of this eliminates the need for engineers. It changes where their time goes — away from the repetitive "reproduce, fix, review" cycle on routine bugs, toward the judgment calls a well-designed system is built to escalate rather than guess at. A system that claims to remove humans from the loop entirely is usually the one you should trust least. Anton Shorygin is a QA Lead with 10+ years in software quality and test automation, currently building automated detection and remediation systems for a production software platform.