Pinning Both Ends: Letting AI Rewrite 15k Lines of Test Tooling An engineer rewrote a personal test-generation tool, Tool Y, from about 15,000 lines to about 3,500 lines in 3.5 working days by pinning both ends — the parsed spreadsheet inputs and the byte-for-byte golden files — and letting AI rewrite the middle, with the engineer reading the AI's code after every round. No old golden file changed except those the upstream specification showed were wrong, and the rewrite replaced hand-written per-element-type classes with a single parser, validator and generator that reads the upstream specification's machine-readable element definitions, so a new specification version becomes a data change. The engineer reported that the old tool's spreadsheet-based test cases could silently change output when a valid row was inserted, since combined cases reference rows by number and the tool only checks that the number is in range. Tool Y, an old test generator, shrank from about 15,000 lines to about 3,500 in 3.5 working days. Not one old golden file changed, except the ones the upstream specification showed were wrong. The method is the same as in Part 1 https://thecloudlet.github.io/technical/refactor-with-ai/part-1-deletion/ : pin both ends, then let the AI rewrite the middle. The input end is the parsed data. The output end is the golden files, byte for byte. The old tool itself was the check, and I read the AI's code after every round. Tool Y was one engineer's personal tool. It reads test specifications from spreadsheets and writes golden files: generated test cases, kept as the known-good set. Another flow, the downstream flow, runs those files as its regression suite. Nobody had maintained tool Y for a long time, so my manager asked me to take it over. The first job was to upgrade it to a newer upstream specification version. The upstream specification is an outside standard. It defines the element types that every test case is built from. Tool Y works, but nobody planned its structure. Every new need added a new script, a new config, or a new copy of an old file. The tree, with names changed: tool-y/ ├── README.md manual post-processing steps ├── config main/ spreadsheets and JSON for the main CLI │ ├── elements.xlsx │ ├── sets.json refers to spreadsheet row numbers │ └── ... ├── gen/ CLI entry point, plus a flag that acts as another │ ├── main.py │ ├── elements.py one hand-written class per element type │ ├── utils.py │ └── golden b.py a second golden path ├── flow/ mode dispatch, called back from gen/ ├── post/ 4 scripts, chained through JSON files ├── config b/ │ └── elements.xlsx same file name as config main/, different content ├── gen b/ generator B: its own entry point, │ ├── main.py and its own copies of elements.py and utils.py │ ├── elements.py │ └── utils.py └── gen c/ generator C: reads a spreadsheet outside the repo A test case can stand alone or combine several others. These are the modes. Each target, one setup of the downstream flow, has its own generator. What I found when I read the code: The worst part was the spreadsheets themselves. Each specification version had its own spreadsheet, and every element type had its own tab in it, far too many to scroll through. The tabs sit along the bottom of the window. Scrolling right through the tabs, I could never reach the one I wanted. Nobody could review a test case in there, let alone maintain one. That scale also hides mistakes. Combined test cases point at spreadsheet rows by number, and the tool checks only that the number is in range. So I tried it. I inserted one valid row and ran the same command again. No warning. The case had the same name, but the golden file was different. The second problem was the generator itself. Every element type is a hand-written class. The upstream specification already publishes machine-readable definitions of its element types, but tool Y never reads them. So a new specification version means editing the classes one by one, by hand. And nothing checks that a class still matches the specification. The upgrade I was asked to do was exactly this work. Patching one script at a time would leave the next owner the same problem. So I rewrote it, under three rules I couldn't break: The goal: all inputs in git, one parser, one validator, and one generator that reads the upstream specification's machine-readable definitions. Then a new specification version is a data change, and a new test case is a recipe, not code. Before I touched any code, I ran the old tool once in every mode and kept the output. These are the old golden files. The new tool has to reproduce them byte for byte. Generators B and C still read inputs from outside git. I ran them as they were, then moved those inputs into git, so every later run starts from the same files. Then I wrote a small compare tool. It takes two output trees and reports every file that differs, down to the field value. CI runs it on every change. First, the spreadsheets needed a replacement. The recipe format had to be easy to read for both a human and a parser, and keep the reason for a case next to the case. And no new dependency just to read a config file. I looked at six: | Format | Comments | Clean git diff | Parser | Catch | |---|---|---|---|---| | Spreadsheet old | no | no | third-party | binary file, references by row number | | JSON | no | yes | stdlib | no comments, so no "why" | | YAML | yes | yes | third-party | guesses types, e.g. 1.10 → 1.1 | | Python as config | yes | yes | it's code | a recipe becomes a program | | Racket DSL | yes | yes | easy to write | few engineers read Racket | | TOML | yes | yes | stdlib 3.11+ | deep nesting gets awkward | YAML reads version: 1.10 as the number 1.1 . I don't want the format to guess. I really wanted Racket, but the next owner would have to maintain my interpreter. So I chose TOML: comments, explicit types, and a parser in the Python standard library. A recipe is one TOML block per test case: Recipes for upstream specification version "1.10". spec version = "1.10" a string, so it stays "1.10" case name = "case 001" test intent = "