{"slug": "ai-coding-tip-037-stop-patching-blind", "title": "AI Coding Tip 037 - Stop Patching Blind", "summary": "A developer argues that AI-generated code patches should never be applied to untested legacy modules without first writing characterization tests, citing a September 2026 Microsoft Excel security update (KB5002914) that silently broke paste and AutoFill operations across Excel 2016 through Microsoft 365 Apps. Microsoft's follow-up fix (KB5002665) traced the failures to workbooks containing conditional formatting, a code path that had never been covered by tests. The guidance recommends recording current behavior, routing AI patches through seams, and rejecting any AI fix that breaks existing tests.", "body_md": "*Old code, new AI, same unwritten test suite.*\n\nTL;DR: Patch code that never had a test written for it, and every quick fix becomes tomorrow's outage.\n\nYou assume a decades old, stable feature must have some kind of safety net, because it has been shipping without complaints for almost twenty years.\n\nAutomated testing wasn't standard practice when much of that code was written.\n\nNobody ever wrote a test for it, not because a team decided to skip that step.\n\nThe step simply didn't exist yet as an expectation.\n\nIn September 2026, Microsoft shipped a security update for Excel, documented on its own [KB5002914 support page](https://support.microsoft.com/en-us/servicing/office/hotfix/excel/5002914), and paste operations started failing silently across Excel 2016, Office 2019, Office 2021 LTSC, Office 2024 LTSC, and Microsoft 365 Apps.\n\n[AutoFill](https://support.microsoft.com/en-us/office/fill-data-automatically-in-worksheet-cells-74e31bdd-d993-45da-aa82-35a236c5b5db) broke too, presumably out of solidarity 🙂.\n\nMicrosoft's follow-up fix, [KB5002665](https://support.microsoft.com/en-us/servicing/office/hotfix/excel/5002665), later confirmed the trigger in its own words: \"If the workbook contains conditional formatting, paste operations might continue to fail.\"\n\nUsers found the pattern before any release note did, flooding [Microsoft's own Q&A forum](https://learn.microsoft.com/en-us/answers/questions/5999281/we-are-facing-issue-in-excel-2026-2019-and-2021-co) with reports of cells stuck mid-selection, borders that kept blinking, and an Escape key that stopped working, more than a hundred replies deep before a real fix shipped.\n\nNobody on that patch team sat down and chose to skip testing the conditional formatting code path.\n\nThere was no test to run, because there had never been one, and a security hotfix isn't the moment anyone stops to write the first test a decades old module has ever had.\n\nThis isn't a Microsoft problem.\n\nIt's what happens anywhere a stable product ships a fix into a module nobody has ever verified with an actual test, at the speed an AI can now generate a diff, against code old enough that testing it was never part of anyone's job.\n\nTreat any code without tests as legacy code, using [Michael Feathers' definition](https://en.wikipedia.org/wiki/Legacy_code) from *Working Effectively with Legacy Code*, no matter how recently it was last edited.\n\nWrite a [characterization test](https://en.wikipedia.org/wiki/Characterization_test) that records the module's current behavior, including the parts you find ugly or confusing, before the AI changes a single line near it.\n\nFind a seam, a place where you can insert a change without editing the original code path, and route the AI's patch through that seam instead of through the untested core.\n\nAsk the AI to draft the missing tests for the exact module you're about to patch, then [review every assertion before you commit](https://dev.to/mcsee/ai-coding-tip-006-review-every-line-before-commit-bmm) to it.\n\nRun the full [regression suite](https://en.wikipedia.org/wiki/Regression_testing) and any [acceptance tests](https://en.wikipedia.org/wiki/Acceptance_testing) tied to the feature before and after the patch, and reject any AI-generated fix that doesn't leave every existing test passing.\n\nScope the change to the smallest edit that closes the actual security hole, and stop the AI from cleaning up nearby legacy code in the same commit.\n\nDon't authorize the AI to change functional behavior while it closes a security hole, and if the fix genuinely can't avoid touching behavior, [make it stop and ask a human](https://dev.to/mcsee/ai-coding-tip-022-give-ai-a-harness-to-work-with-274a) before it does.\n\nAdd every new characterization test to [the pipeline](https://dev.to/mcsee/ai-coding-tip-032-build-a-dark-factory-pipeline-9b5), so the same regression under the same conditions never again depends on a user noticing it first.\n\nShip the patch to a small [canary](https://martinfowler.com/bliki/CanaryRelease.html) segment before a full rollout, so a blind spot in decades old code surfaces on a fraction of your users instead of on all of them at once.\n\nA canary isn't always possible with a security hole.\n\nWhen the vulnerability is already public or actively exploited, leaving any segment unpatched just hands the remaining users to whoever is exploiting it, so ship to everyone at once and lean on the characterization tests instead of a staged rollout to catch a regression.\n\n**Ship with evidence:** A passing characterization suite replaces a shrug and a hope with [a specific, checkable claim](https://dev.to/mcsee/ai-coding-tip-006-review-every-line-before-commit-bmm) about what still works.\n\n**Shrink the blast radius:** A canary rollout turns a legacy blind spot into a contained incident instead of a headline about broken spreadsheets.\n\n**Turn legacy code into safe territory:** Characterization tests convert unknown behavior into known behavior, so the AI can extend it without guessing.\n\n**Stop the firefighting cycle:** A tested seam absorbs the next patch too, instead of forcing another blind edit into the same fragile module.\n\n**Protect a stable product's reputation:** Users forgive a slow fix; they don't forgive a security update that quietly [breaks](https://dev.to/mcsee/stop-calling-them-bugs-57gl) a feature they use every day.\n\n**Make review possible:** A reviewer can [check a diff against a test](https://dev.to/mcsee/ai-coding-tip-024-force-a-criteria-check-before-the-task-ends-51ij) far faster than they can trace decades old logic by eye.\n\nMichael Feathers opened *Working Effectively with Legacy Code* with a blunt definition: legacy code is simply code without tests.\n\nNot old code, not ugly code, code you can't safely change because nothing tells you when you broke it.\n\nExcel's conditional formatting and clipboard logic fits that definition perfectly, and so does most of the code every long-lived product still runs in production.\n\nAge alone doesn't make that inevitable, no matter how much a rushed patch team would like to blame the calendar.\n\nMeanwhile, over at [SQLite](https://www.sqlite.org/mostdeployed.html), somebody has clearly never let a deadline win an argument: running since August 2000, sitting inside an estimated one trillion database files worldwide, and likely the most widely deployed software library on the planet after zlib.\n\nIts [own testing page](https://www.sqlite.org/testing.html) states the project carries roughly 590 times as much test code as product code: fuzzing that runs about a billion mutations a day, out-of-memory and I/O fault injection, crash recovery checks, and a claimed 100% branch and MC/DC coverage on the core engine, verified under Valgrind and multiple sanitizers before every release.\n\nTwenty-five years old and drowning in tests is a choice a team made and kept making.\n\nTwenty-five years old and never tested once is a different choice, made by omission a long time ago and repeated every time nobody goes back to fix it.\n\nAI changes the economics of that second choice in a way Feathers never had to plan for.\n\nWriting a patch used to be the slow part, and testing it thoroughly, even manually, could roughly keep pace.\n\nNow an AI assistant can [draft a fix](https://dev.to/mcsee/ai-coding-tip-023-shrink-your-ais-pull-request-4lnb) for a fifteen year old code path in the time it takes to read the CVE, and the pace of change [outruns the pace of verification](https://dev.to/mcsee/ai-coding-tip-021-avoid-comprehension-debt-edm) unless the test suite is automated and runs on every change.\n\nA characterization test doesn't ask whether the old behavior was correct.\n\nIt asks whether the new code still does what the old code did, on purpose or not, so a patch can prove it didn't change anything it wasn't supposed to touch.\n\nThat's the seam Feathers describes: a point in the code where you can alter behavior without editing the class or function that currently owns it, which is exactly where an AI-generated fix belongs when the surrounding code has no coverage.\n\nFeathers built his whole book around getting legacy code under test through seams like this one, not around rewriting the module first and hoping the tests catch up later.\n\nA security patch through a seam only touches the one behavior the CVE is about.\n\nOnly [a passing regression suite in the exit criteria](https://dev.to/mcsee/ai-coding-tip-024-force-a-criteria-check-before-the-task-ends-51ij) counts, not just code that compiles and looks plausible.\n\n[Code standards you force on every change](https://dev.to/mcsee/ai-coding-tip-027-force-code-standards-58nf) should include this one explicitly for any file that touches a stable, shipped feature.\n\nAnd when an AI reports that tests pass, verify it the way you'd [verify any other AI claim you can't fully see](https://dev.to/mcsee/ai-coding-tip-033-protect-yourself-against-ai-cheating-1pn1): run the suite yourself, don't take the summary as proof.\n\nThe KB5002914 incident didn't need a smarter model to avoid.\n\nIt needed a characterization test on the conditional formatting code path, run automatically before the patch shipped to hundreds of millions of installs.\n\nUsers get frustrated by new features that don't work, but they never tolerate [breaking changes](https://dev.to/mcsee/code-smell-303-breaking-changes-36hh) on functionality they've been using for years.\n\n```\nFix the security hole \nin the conditional formatting module and ship it today.\nThis code has never had a test.\nIt has worked fine for years without one.\nShip it the same way.\nWrite characterization tests \nFor the conditional formatting module.\n\nIt never had tests.\nYou must capture its real clipboard and AutoFill behavior.\n\nShow me the failing diff\nOnly after every existing and new test passes.\n\nI will also validate it manually.\n\nScope the fix to \nThe smallest change that closes the security hole.\n\nYou aren't authorized to change functional behavior.\n\nIf you must, stop and ask me first.\n```\n\nA characterization test doesn't judge whether old behavior was good design.\n\nIt only proves the patch didn't silently change it, which is a different and more urgent question during an emergency fix.\n\nWriting that first test on truly untested legacy code takes real hours you don't have during an active incident, so the honest move is budgeting for it before the next CVE, not during it, while everyone is yelling in the incident channel.\n\nAn AI can draft the test scaffolding quickly, but someone still has to confirm the assertions describe the behavior you actually want preserved, not the behavior the AI assumed from the function name.\n\nA security patch authorizes closing the hole, nothing else.\n\nIf the AI decides the smallest fix still requires changing what the feature does, [that decision belongs to a human](https://dev.to/mcsee/ai-coding-tip-015-force-the-ai-to-obey-you-49mc), not to whichever model happened to draft the diff.\n\nA canary rollout only limits damage if someone is watching it, so pair it with the same kind of monitoring you'd want on any [privileged system change](https://dev.to/mcsee/ai-coding-tip-036-grant-ai-the-least-privilege-possible-1491).\n\n[X] Semi-Automatic\n\nCharacterization tests describe current behavior, not correct behavior, so a [defect](https://dev.to/mcsee/stop-calling-them-bugs-57gl) baked into the legacy code gets preserved right alongside everything that works, immortalized by the very tests meant to protect you.\n\nRetrofitting tests onto a codebase with none takes longer than the emergency patch itself, which is exactly why teams skip it under deadline pressure, and exactly why the skipping keeps happening, generation after generation of engineers inheriting the same untested pile and pretending it's fine.\n\n[X] Intermediate\n\nEvery stable product is one untested module away from becoming a headline about broken copy and paste.\n\nAI didn't create that risk, but it lets a team reach it faster than ever, one confidently drafted patch at a time, cheerfully unaware it just walked into a minefield nobody mapped.\n\nWrite the characterization test first, find the seam, and let the AI work inside walls a human actually verified.\n\nA quick fix that breaks a feature everyone relies on isn't quick.\n\nIt's just a slower disaster with better timing.\n\n[KB5002914 Support Page, Microsoft](https://support.microsoft.com/en-us/servicing/office/hotfix/excel/5002914)\n\n[KB5002665 Support Page, Microsoft](https://support.microsoft.com/en-us/servicing/office/hotfix/excel/5002665)\n\n[We Are Facing Issue in Excel 2026, 2019 and 2021, Microsoft Q&A](https://learn.microsoft.com/en-us/answers/questions/5999281/we-are-facing-issue-in-excel-2026-2019-and-2021-co)\n\n[Microsoft Office September 8, 2026 Update Causes Excel Copy Paste Issue, Born City](https://borncity.com/win/2026/09/10/microsoft-office-september-8-2026-update-causes-excel-copy-paste-issue/)\n\n[Most Widely Deployed and Used Database Engine, SQLite](https://www.sqlite.org/mostdeployed.html)\n\n[Characterization Test, Wikipedia](https://en.wikipedia.org/wiki/Characterization_test)\n\nAny unit testing framework already in your stack, a [feature flag](https://en.wikipedia.org/wiki/Feature_toggle) or canary deployment tool, and a coverage reporter wired into continuous integration so a missing test on a changed file blocks the merge.\n\nThe views expressed here are my own.\n\nI am a human who writes as best as possible for other humans.\n\nI use AI proofreading tools to improve some texts.\n\nMost AI detectors will flag this article as AI-generated. That's expected. It's a technical article. It has a rigid format and clear steps to follow.\n\nThat's exactly the pattern those tools are trained to catch. I've apparently been \"writing like an AI\" for decades, long before AI existed. This is a technical article, not a novel.\n\nI welcome constructive criticism and dialogue.\n\nI shape these insights through 30 years in the software industry, 25 years of teaching, and writing over 500 articles and a book.\n\nThis article is part of the *AI Coding Tip* series.", "url": "https://wpnews.pro/news/ai-coding-tip-037-stop-patching-blind", "canonical_source": "https://dev.to/mcsee/ai-coding-tip-037-stop-patching-blind-227f", "published_at": "2026-09-22 11:00:00+00:00", "updated_at": "2026-09-22 11:23:13.667247+00:00", "lang": "en", "topics": ["ai-tools", "developer-tools", "ai-products"], "entities": ["Microsoft", "Excel", "KB5002914", "KB5002665", "Michael Feathers", "Working Effectively with Legacy Code"], "alternates": {"html": "https://wpnews.pro/news/ai-coding-tip-037-stop-patching-blind", "markdown": "https://wpnews.pro/news/ai-coding-tip-037-stop-patching-blind.md", "text": "https://wpnews.pro/news/ai-coding-tip-037-stop-patching-blind.txt", "jsonld": "https://wpnews.pro/news/ai-coding-tip-037-stop-patching-blind.jsonld"}}