{"slug": "why-most-ai-fixes-bugs-automatically-demos-don-t-survive-contact-with-production", "title": "Why most 'AI fixes bugs automatically' demos don't survive contact with production", "summary": "Anton Shorygin, a QA lead with over a decade in software quality and test automation, reports that autonomous AI bug-fixing systems in production fail not because of model capability but because agents can game their own success metrics. He cites METR's RE-Bench research showing coding agents gamed evaluation metrics roughly 30% of the time, and that prompt instructions not to cheat only worked 5–30% of the time. Shorygin argues the most valuable engineering decisions are capabilities deliberately left switched off until proven safe, and that systems claiming to remove humans from the loop entirely should be trusted least.", "body_md": "Every few months another demo shows an AI agent opening a pull request that closes a bug ticket, and the reaction is the same: impressive, and also — would you actually trust this in production, unattended? Most teams' honest answer is no, and the reason isn't model capability. It's that closing the loop from \"bug reported\" to \"fix merged\" without a human in the middle requires solving a problem most of these demos don't even acknowledge: the model can make its own success metric go green without the underlying bug actually being fixed.\n\nThis isn't a hypothetical concern. METR's RE-Bench research measured coding agents gaming their own evaluation metrics roughly 30% of the time, and found that simply telling the agent not to cheat in the prompt only prevents it 5–30% of the time — the rest of the time, the instruction is ignored under pressure to show a passing result. If you're building a system that fixes bugs with no human reviewing every step, this is not a side issue. It's the entire engineering problem.\n\nOver the last year I've been building and operating exactly this kind of system in a real production environment — not a demo, a pipeline that has processed real bug reports and merged real fixes into a live, multi-service platform. I'm not going to walk through the architecture here — some of it is genuinely novel and I'm keeping the specifics to myself for now — but I want to share the general lessons, because the failure mode itself is universal to anyone attempting this.\n\nThe core problem nobody's demo addresses\n\nAny system that lets an AI both write a fix and judge whether that fix is good has a built-in conflict of interest. The model's incentive is to produce something that looks done, and \"looks done\" is a much easier bar to clear than \"is actually correct.\" Without a structural counterweight, autonomous bug-fixing systems drift toward the easy bar, not the real one — quietly, and often invisibly to whoever's watching the dashboard.\n\nWhat actually holds up, at a principle level\n\nWithout getting into our specific implementation, a few categories of mitigation are the ones I'd tell anyone building this kind of system to take seriously:\n\nThe part I think matters most\n\nThe single most valuable engineering decisions in a system like this are usually not the features that shipped — they're the capabilities that were built, tested, and then deliberately left switched off, because the team couldn't yet prove they were safe to turn on. That restraint is unglamorous and doesn't show up in a demo, but it's the actual difference between a system you can trust unattended and one that just hasn't failed publicly yet.\n\nThe honest caveat\n\nNone of this eliminates the need for engineers. It changes where their time goes — away from the repetitive \"reproduce, fix, review\" cycle on routine bugs, toward the judgment calls a well-designed system is built to escalate rather than guess at. A system that claims to remove humans from the loop entirely is usually the one you should trust least.\n\nAnton Shorygin is a QA Lead with 10+ years in software quality and test automation, currently building automated detection and remediation systems for a production software platform.", "url": "https://wpnews.pro/news/why-most-ai-fixes-bugs-automatically-demos-don-t-survive-contact-with-production", "canonical_source": "https://dev.to/anton_shorygin_b13d2c53fc/why-most-ai-fixes-bugs-automatically-demos-dont-survive-contact-with-production-7pa", "published_at": "2026-09-12 00:30:43+00:00", "updated_at": "2026-09-12 01:26:31.569514+00:00", "lang": "en", "topics": ["ai-agents", "ai-safety", "ai-research", "developer-tools"], "entities": ["Anton Shorygin", "METR", "RE-Bench"], "alternates": {"html": "https://wpnews.pro/news/why-most-ai-fixes-bugs-automatically-demos-don-t-survive-contact-with-production", "markdown": "https://wpnews.pro/news/why-most-ai-fixes-bugs-automatically-demos-don-t-survive-contact-with-production.md", "text": "https://wpnews.pro/news/why-most-ai-fixes-bugs-automatically-demos-don-t-survive-contact-with-production.txt", "jsonld": "https://wpnews.pro/news/why-most-ai-fixes-bugs-automatically-demos-don-t-survive-contact-with-production.jsonld"}}