Pin the Entrypoint Snapshot Before Any Internal Refactor

A developer advises locking an entrypoint's observable outputs before refactoring internal code, warning that helper tests miss script-level behavior such as print text, file bytes, and CLI argument order. The post uses a teaching fixture to demonstrate building golden snapshots from frozen inputs and re-running the full command after each edit.

A messy-repo refactor fails when helpers move first. Lock the entrypoint's observable outputs before any edit. Then change one internal function and nothing else. Most AI patches target a small function in isolation. That function often looks cleaner after the edit. Callers still depend on files, globals, and print order. A passing helper test does not protect the script. The script is what operators actually run. Treat that script as the contract, not the helper. Cheap model output makes large diffs easy to produce. It does not make those diffs safe to merge. Safety still comes from frozen, replayable observables. The listing below is a labeled teaching fixture. It is not production code from a live system. It mixes pricing, tax, and invoice file writes. python messy invoice.py — teaching fixture, not production from pathlib import Path import json import sys TAX = 0.08 OUT = Path "out" last = {} def load items path : rows = for line in Path path .read text .splitlines : sku, qty, price = line.split "," rows.append {"sku": sku, "qty": int qty , "price": float price } last "rows" = rows return rows def subtotal rows : s = 0.0 for r in rows: s += r "qty" r "price" if r "qty" = 10: s -= r "price" implicit bulk rule last "subtotal" = s return s def tax on amount, region : rate = TAX if region == "EU": rate = 0.19 if region == "EXEMPT": rate = 0.0 return round amount rate, 2 def write invoice rows, region, dest : OUT.mkdir exist ok=True sub = subtotal rows tax = tax on sub, region total = round sub + tax, 2 payload = { "region": region, "lines": len rows , "subtotal": sub, "tax": tax, "total": total, } Path dest .write text json.dumps payload, indent=2 + "\n" print f"WROTE {dest} total={total}" return payload def main argv : src = argv 1 region = argv 2 dest = argv 3 rows = load items src return write invoice rows, region, dest if name == " main ": main sys.argv Three hazards sit inside that short teaching module. The global last dict stores implicit process state. The subtotal loop hides a bulk discount rule. Helper tests miss print text and file bytes. They also miss argument order on the CLI. A whole-run snapshot catches those observables in one gate. Build a golden directory from a frozen input corpus. Store stdout, stderr, exit code, and output hashes. Re-run the same command after every internal edit. Keep the input fixtures tiny and committed to git. One CSV is enough for the first gate. Add a second CSV only after the first stays green. fixtures/items basic.csv A,1,10.00 B,10,2.50 C,3,4.00 fixtures/items eu.csv D,2,40.00 E,10,1.00 Hand totals keep the first goldens honest. Do not trust the script to mark its own exam. The US basic case should resolve as follows. Line A contributes 1 10.00 = 10.00 . Line B contributes 10 2.50 = 25.00 . Quantity 10 then subtracts 2.50 as bulk credit. Line C contributes 3 4.00 = 12.00 . Subtotal is 10.00 + 22.50 + 12.00 = 44.50 . US tax is round 44.50 0.08, 2 = 3.56 . Total is round 44.50 + 3.56, 2 = 48.06 . Stdout must read WROTE out/inv.json total=48.06 . Any later extract must preserve those exact bytes. The EU bulk case should resolve next. Line D contributes 2 40.00 = 80.00 . Line E contributes 10 1.00 - 1.00 = 9.00 . Subtotal is 89.00 before region tax. EU tax is round 89.00 0.19, 2 = 16.91 . Total is round 89.00 + 16.91, 2 = 105.91 . Run the entrypoint under a clean working directory. Do not reuse leftover output files between cases. Capture the process, not a Python function call. python char harness.py — teaching fixture from future import annotations import hashlib import json import os import shutil import subprocess import sys from pathlib import Path ROOT = Path file .resolve .parent GOLDEN = ROOT / "golden" CASES = { "name": "us basic", "args": "fixtures/items basic.csv", "US", "out/inv.json" , }, { "name": "eu bulk", "args": "fixtures/items eu.csv", "EU", "out/inv.json" , }, { "name": "exempt basic", "args": "fixtures/items basic.csv", "EXEMPT", "out/inv.json" , }, def sha256 path: Path - str | None: if not path.is file : return None h = hashlib.sha256 h.update path.read bytes return h.hexdigest def run case case: dict - dict: work = ROOT / "work" / case "name" if work.exists : shutil.rmtree work work.mkdir parents=True shutil.copytree ROOT / "fixtures", work / "fixtures" dest rel = Path case "args" 2 proc = subprocess.run sys.executable, str ROOT / "messy invoice.py" , case "args" , cwd=work, capture output=True, text=True, env={ os.environ, "PYTHONHASHSEED": "0"}, out file = work / dest rel return { "name": case "name" , "exit code": proc.returncode, "stdout": proc.stdout, "stderr": proc.stderr, "out sha256": sha256 out file , "out text": out file.read text if out file.is file else None, } def record - None: GOLDEN.mkdir exist ok=True for case in CASES: snap = run case case GOLDEN / f"{case 'name' }.json" .write text json.dumps snap, indent=2 + "\n" print f"recorded {case 'name' }" def check - int: failed = 0 for case in CASES: got = run case case path = GOLDEN / f"{case 'name' }.json" want = json.loads path.read text keys = "exit code", "stdout", "stderr", "out sha256", "out text" for key in keys: if got key = want key : failed += 1 print f"DRIFT {case 'name' } {key}" print f" want={want key r}" print f" got ={got key r}" if failed: print f"{failed} field s drifted" return 1 print "snapshot gate green" return 0 if name == " main ": cmd = sys.argv 1 if len sys.argv 1 else "check" if cmd == "record": record else: raise SystemExit check Each case records five comparable fields on disk. Those fields are the only pass signal. Pretty logs outside the snapshot are noise. A gate that never fails is not a gate. Break one print on purpose after recording. Confirm that check reports DRIFT on stdout. python char harness.py record python char harness.py check Edit the print format, then run check again. Restore the print before any real refactor. The restored run must return exit code 0. A sample drift block looks like this. The numbers below match a one-cent rounding slip. Treat that slip as a failed extract, not noise. DRIFT us basic stdout want='WROTE out/inv.json total=48.06\n' got ='WROTE out/inv.json total=48.06\n' DRIFT us basic out text want='{\n "region": "US",\n "lines": 3,\n "subtotal": 44.5,\n "tax": 3.56,\n "total": 48.06\n}\n' got ='{\n "region": "US",\n "lines": 3,\n "subtotal": 44.5,\n "tax": 3.56,\n "total": 48.05\n}\n' Do not rename files in the same patch. Do not move CLI flags in the same patch. Do not retune tax rounding in the same patch. The smallest safe change in this module is extraction. Pull the bulk rule out of subtotal . Keep write invoice output byte-identical after the extract. python def bulk credit row : if row "qty" = 10: return row "price" return 0.0 def subtotal rows : s = 0.0 for r in rows: s += r "qty" r "price" s -= bulk credit r last "subtotal" = s return s Re-run the snapshot gate after that extract. Green means the entrypoint still writes the same bytes. Red means the extract changed pricing or print text. python char harness.py check once. Skip any patch that fails one checklist row. Wide cleanup is not a first-cycle goal. Queue extra extracts for later green cycles. | Drift field | Likely cause | Safe action | |---|---|---| | exit code | uncaught exception or new sys.exit | stop; inspect traceback | | stdout | print format or call order | stop unless format was the goal | | stderr | new warning or log line | treat as contract unless documented | | out sha256 | numeric rounding or key order | stop; compare out text | | out text | tax, bulk rule, or region mapping | revert; split the change | Use one row as a stop rule, not a suggestion. Any unexplained drift must block the current patch. Do not rewrite goldens to match a guessed refactor. Invoice totals look like business rules, not formatting. A one-cent drift is a failed extract. Do not round-trip floats through new types in the same patch. Do not switch json.dumps settings during extract. Indent, separators, and key order are contract bytes. Hash equality will fail if those settings move. The working directory leaks into relative output paths. Set PYTHONHASHSEED to keep hash walks stable. Copy fixtures into a fresh work tree every case. Path separators can drift across operating systems. Keep dest arguments in POSIX form inside cases. Run the gate on one OS, not two, per corpus. Grow new cases only from escaped production bugs. This teaching fixture records only three named cases. Those three cases will still miss many branches. Add a case when a real bug escapes, not before. Each new case must fail once before it is recorded. A case that never failed is an untested assertion. The coding model proposes the internal extract only. The snapshot gate accepts or rejects that extract. Do not let the model refresh golden files. Disclosure: This article was prepared as part of MonkeyCode's product outreach. MonkeyCode offers free model access and a free server option. Those two options can host this harness beside the messy module. The model sees the frozen cases and the current function body. It does not get permission to rewrite golden snapshots. Keep the model prompt narrow, mechanical, and single-purpose. The block below is an unexecuted prompt template. It is not a log from a real session. Paste it only after check is already green. Preserve golden/ byte-for-byte. Do not edit fixtures, CLI args, or char harness.py. Extract bulk credit from subtotal in messy invoice.py. Return one patch. Stop if any snapshot field would drift. Ask for one extract that preserves snapshot bytes. Reject patches that touch fixtures, goldens, or CLI args. If you try that workflow, run the harness locally first. Then point a free MonkeyCode session at the same gate. It does not prove thread safety or performance. It does not prove unknown regions or empty files. It does not prove tax law, only current bytes. Hash equality is brittle with unstable key order. JSON dumps must keep stable separators and indent. Timestamps inside invoices will break this design. Hidden network calls will also escape this gate. So will clock reads and unordered set iteration. Strip those sources before recording the first corpus. Skip this if you still lack a runnable entrypoint. Skip this if outputs include raw secrets or PII. Skip this if the script is nondeterministic by design. Skip this for greenfield modules with no users. Those modules need designed tests, not snapshots. Characterization is for behavior you cannot rewrite from memory. Do not use this as a license for large rewrites. The method allows one internal change per cycle. Wide cleanups belong after many green cycles, not before. Start at the script the operator actually runs. Record stdout, exit code, and output file hashes. Extract one helper only after that gate is green.