AI Coding Tip 037 - Stop Patching Blind

A developer argues that AI-generated code patches should never be applied to untested legacy modules without first writing characterization tests, citing a September 2026 Microsoft Excel security update (KB5002914) that silently broke paste and AutoFill operations across Excel 2016 through Microsoft 365 Apps. Microsoft's follow-up fix (KB5002665) traced the failures to workbooks containing conditional formatting, a code path that had never been covered by tests. The guidance recommends recording current behavior, routing AI patches through seams, and rejecting any AI fix that breaks existing tests.

Old code, new AI, same unwritten test suite. TL;DR: Patch code that never had a test written for it, and every quick fix becomes tomorrow's outage. You assume a decades old, stable feature must have some kind of safety net, because it has been shipping without complaints for almost twenty years. Automated testing wasn't standard practice when much of that code was written. Nobody ever wrote a test for it, not because a team decided to skip that step. The step simply didn't exist yet as an expectation. In September 2026, Microsoft shipped a security update for Excel, documented on its own KB5002914 support page https://support.microsoft.com/en-us/servicing/office/hotfix/excel/5002914 , and paste operations started failing silently across Excel 2016, Office 2019, Office 2021 LTSC, Office 2024 LTSC, and Microsoft 365 Apps. AutoFill https://support.microsoft.com/en-us/office/fill-data-automatically-in-worksheet-cells-74e31bdd-d993-45da-aa82-35a236c5b5db broke too, presumably out of solidarity 🙂. Microsoft's follow-up fix, KB5002665 https://support.microsoft.com/en-us/servicing/office/hotfix/excel/5002665 , later confirmed the trigger in its own words: "If the workbook contains conditional formatting, paste operations might continue to fail." Users found the pattern before any release note did, flooding Microsoft's own Q&A forum https://learn.microsoft.com/en-us/answers/questions/5999281/we-are-facing-issue-in-excel-2026-2019-and-2021-co with reports of cells stuck mid-selection, borders that kept blinking, and an Escape key that stopped working, more than a hundred replies deep before a real fix shipped. Nobody on that patch team sat down and chose to skip testing the conditional formatting code path. There was no test to run, because there had never been one, and a security hotfix isn't the moment anyone stops to write the first test a decades old module has ever had. This isn't a Microsoft problem. It's what happens anywhere a stable product ships a fix into a module nobody has ever verified with an actual test, at the speed an AI can now generate a diff, against code old enough that testing it was never part of anyone's job. Treat any code without tests as legacy code, using Michael Feathers' definition https://en.wikipedia.org/wiki/Legacy code from Working Effectively with Legacy Code , no matter how recently it was last edited. Write a characterization test https://en.wikipedia.org/wiki/Characterization test that records the module's current behavior, including the parts you find ugly or confusing, before the AI changes a single line near it. Find a seam, a place where you can insert a change without editing the original code path, and route the AI's patch through that seam instead of through the untested core. Ask the AI to draft the missing tests for the exact module you're about to patch, then review every assertion before you commit https://dev.to/mcsee/ai-coding-tip-006-review-every-line-before-commit-bmm to it. Run the full regression suite https://en.wikipedia.org/wiki/Regression testing and any acceptance tests https://en.wikipedia.org/wiki/Acceptance testing tied to the feature before and after the patch, and reject any AI-generated fix that doesn't leave every existing test passing. Scope the change to the smallest edit that closes the actual security hole, and stop the AI from cleaning up nearby legacy code in the same commit. Don't authorize the AI to change functional behavior while it closes a security hole, and if the fix genuinely can't avoid touching behavior, make it stop and ask a human https://dev.to/mcsee/ai-coding-tip-022-give-ai-a-harness-to-work-with-274a before it does. Add every new characterization test to the pipeline https://dev.to/mcsee/ai-coding-tip-032-build-a-dark-factory-pipeline-9b5 , so the same regression under the same conditions never again depends on a user noticing it first. Ship the patch to a small canary https://martinfowler.com/bliki/CanaryRelease.html segment before a full rollout, so a blind spot in decades old code surfaces on a fraction of your users instead of on all of them at once. A canary isn't always possible with a security hole. When the vulnerability is already public or actively exploited, leaving any segment unpatched just hands the remaining users to whoever is exploiting it, so ship to everyone at once and lean on the characterization tests instead of a staged rollout to catch a regression. Ship with evidence: A passing characterization suite replaces a shrug and a hope with a specific, checkable claim https://dev.to/mcsee/ai-coding-tip-006-review-every-line-before-commit-bmm about what still works. Shrink the blast radius: A canary rollout turns a legacy blind spot into a contained incident instead of a headline about broken spreadsheets. Turn legacy code into safe territory: Characterization tests convert unknown behavior into known behavior, so the AI can extend it without guessing. Stop the firefighting cycle: A tested seam absorbs the next patch too, instead of forcing another blind edit into the same fragile module. Protect a stable product's reputation: Users forgive a slow fix; they don't forgive a security update that quietly breaks https://dev.to/mcsee/stop-calling-them-bugs-57gl a feature they use every day. Make review possible: A reviewer can check a diff against a test https://dev.to/mcsee/ai-coding-tip-024-force-a-criteria-check-before-the-task-ends-51ij far faster than they can trace decades old logic by eye. Michael Feathers opened Working Effectively with Legacy Code with a blunt definition: legacy code is simply code without tests. Not old code, not ugly code, code you can't safely change because nothing tells you when you broke it. Excel's conditional formatting and clipboard logic fits that definition perfectly, and so does most of the code every long-lived product still runs in production. Age alone doesn't make that inevitable, no matter how much a rushed patch team would like to blame the calendar. Meanwhile, over at SQLite https://www.sqlite.org/mostdeployed.html , somebody has clearly never let a deadline win an argument: running since August 2000, sitting inside an estimated one trillion database files worldwide, and likely the most widely deployed software library on the planet after zlib. Its own testing page https://www.sqlite.org/testing.html states the project carries roughly 590 times as much test code as product code: fuzzing that runs about a billion mutations a day, out-of-memory and I/O fault injection, crash recovery checks, and a claimed 100% branch and MC/DC coverage on the core engine, verified under Valgrind and multiple sanitizers before every release. Twenty-five years old and drowning in tests is a choice a team made and kept making. Twenty-five years old and never tested once is a different choice, made by omission a long time ago and repeated every time nobody goes back to fix it. AI changes the economics of that second choice in a way Feathers never had to plan for. Writing a patch used to be the slow part, and testing it thoroughly, even manually, could roughly keep pace. Now an AI assistant can draft a fix https://dev.to/mcsee/ai-coding-tip-023-shrink-your-ais-pull-request-4lnb for a fifteen year old code path in the time it takes to read the CVE, and the pace of change outruns the pace of verification https://dev.to/mcsee/ai-coding-tip-021-avoid-comprehension-debt-edm unless the test suite is automated and runs on every change. A characterization test doesn't ask whether the old behavior was correct. It asks whether the new code still does what the old code did, on purpose or not, so a patch can prove it didn't change anything it wasn't supposed to touch. That's the seam Feathers describes: a point in the code where you can alter behavior without editing the class or function that currently owns it, which is exactly where an AI-generated fix belongs when the surrounding code has no coverage. Feathers built his whole book around getting legacy code under test through seams like this one, not around rewriting the module first and hoping the tests catch up later. A security patch through a seam only touches the one behavior the CVE is about. Only a passing regression suite in the exit criteria https://dev.to/mcsee/ai-coding-tip-024-force-a-criteria-check-before-the-task-ends-51ij counts, not just code that compiles and looks plausible. Code standards you force on every change https://dev.to/mcsee/ai-coding-tip-027-force-code-standards-58nf should include this one explicitly for any file that touches a stable, shipped feature. And when an AI reports that tests pass, verify it the way you'd verify any other AI claim you can't fully see https://dev.to/mcsee/ai-coding-tip-033-protect-yourself-against-ai-cheating-1pn1 : run the suite yourself, don't take the summary as proof. The KB5002914 incident didn't need a smarter model to avoid. It needed a characterization test on the conditional formatting code path, run automatically before the patch shipped to hundreds of millions of installs. Users get frustrated by new features that don't work, but they never tolerate breaking changes https://dev.to/mcsee/code-smell-303-breaking-changes-36hh on functionality they've been using for years. Fix the security hole in the conditional formatting module and ship it today. This code has never had a test. It has worked fine for years without one. Ship it the same way. Write characterization tests For the conditional formatting module. It never had tests. You must capture its real clipboard and AutoFill behavior. Show me the failing diff Only after every existing and new test passes. I will also validate it manually. Scope the fix to The smallest change that closes the security hole. You aren't authorized to change functional behavior. If you must, stop and ask me first. A characterization test doesn't judge whether old behavior was good design. It only proves the patch didn't silently change it, which is a different and more urgent question during an emergency fix. Writing that first test on truly untested legacy code takes real hours you don't have during an active incident, so the honest move is budgeting for it before the next CVE, not during it, while everyone is yelling in the incident channel. An AI can draft the test scaffolding quickly, but someone still has to confirm the assertions describe the behavior you actually want preserved, not the behavior the AI assumed from the function name. A security patch authorizes closing the hole, nothing else. If the AI decides the smallest fix still requires changing what the feature does, that decision belongs to a human https://dev.to/mcsee/ai-coding-tip-015-force-the-ai-to-obey-you-49mc , not to whichever model happened to draft the diff. A canary rollout only limits damage if someone is watching it, so pair it with the same kind of monitoring you'd want on any privileged system change https://dev.to/mcsee/ai-coding-tip-036-grant-ai-the-least-privilege-possible-1491 . X Semi-Automatic Characterization tests describe current behavior, not correct behavior, so a defect https://dev.to/mcsee/stop-calling-them-bugs-57gl baked into the legacy code gets preserved right alongside everything that works, immortalized by the very tests meant to protect you. Retrofitting tests onto a codebase with none takes longer than the emergency patch itself, which is exactly why teams skip it under deadline pressure, and exactly why the skipping keeps happening, generation after generation of engineers inheriting the same untested pile and pretending it's fine. X Intermediate Every stable product is one untested module away from becoming a headline about broken copy and paste. AI didn't create that risk, but it lets a team reach it faster than ever, one confidently drafted patch at a time, cheerfully unaware it just walked into a minefield nobody mapped. Write the characterization test first, find the seam, and let the AI work inside walls a human actually verified. A quick fix that breaks a feature everyone relies on isn't quick. It's just a slower disaster with better timing. KB5002914 Support Page, Microsoft https://support.microsoft.com/en-us/servicing/office/hotfix/excel/5002914 KB5002665 Support Page, Microsoft https://support.microsoft.com/en-us/servicing/office/hotfix/excel/5002665 We Are Facing Issue in Excel 2026, 2019 and 2021, Microsoft Q&A https://learn.microsoft.com/en-us/answers/questions/5999281/we-are-facing-issue-in-excel-2026-2019-and-2021-co Microsoft Office September 8, 2026 Update Causes Excel Copy Paste Issue, Born City https://borncity.com/win/2026/09/10/microsoft-office-september-8-2026-update-causes-excel-copy-paste-issue/ Most Widely Deployed and Used Database Engine, SQLite https://www.sqlite.org/mostdeployed.html Characterization Test, Wikipedia https://en.wikipedia.org/wiki/Characterization test Any unit testing framework already in your stack, a feature flag https://en.wikipedia.org/wiki/Feature toggle or canary deployment tool, and a coverage reporter wired into continuous integration so a missing test on a changed file blocks the merge. The views expressed here are my own. I am a human who writes as best as possible for other humans. I use AI proofreading tools to improve some texts. Most AI detectors will flag this article as AI-generated. That's expected. It's a technical article. It has a rigid format and clear steps to follow. That's exactly the pattern those tools are trained to catch. I've apparently been "writing like an AI" for decades, long before AI existed. This is a technical article, not a novel. I welcome constructive criticism and dialogue. I shape these insights through 30 years in the software industry, 25 years of teaching, and writing over 500 articles and a book. This article is part of the AI Coding Tip series.