{"slug": "snapshot-file-deltas-before-you-extract-glue", "title": "Snapshot File Deltas Before You Extract Glue", "summary": "A developer proposes a characterization-testing approach for refactoring untested 'glue' code that writes files, reads environment variables, or changes directories. The method records a frozen ledger of process behavior—exit code, stdout, stderr, file deltas, and environment keys—before any extraction, to prevent hidden side effects from breaking behavior. The post includes example code for a report script and a harness to capture such ledgers.", "body_md": "Do not extract glue until side effects are pinned. Messy modules leak behavior through files, env, and cwd. A function signature is not a characterization test.\n\nAI diffs look clean when tests are missing. Glue code still writes caches, logs, and temp files. Reviewers then ship a rename that changes process behavior.\n\nCheap code generation does not reduce the characterization cost. Technical debt remains in implicit state, not syntax. You need a ledger before the first extract.\n\nUntested glue often reads hidden config from the process. It may chdir, mkdir, or rewrite a lockfile. Those actions never appear in the public return value.\n\nReturn values also miss mkdir side effects on empty input. Missing directories can change later commands in the same job. That is still user-visible behavior, not an internal detail.\n\nA ledger is a frozen record of process behavior. It is not a unit test of internal helpers. It pins observable results for a fixed fixture pack.\n\nRecord five observable channels on every fixture run. Capture exit code, stdout, stderr, file deltas, and env keys. Ignore wall-clock time unless the contract requires it.\n\nHash file contents, not only the file names. Store relative paths instead of absolute sandbox paths. Drop volatile fields like pid, dates, and hostnames.\n\nNormalize stdout by stripping any sandbox path prefix. Keep the newline style identical across all runs. Sort JSON object keys before hashing cache files.\n\nConsider a report glue script with no tests. It reads CSV from stdin and writes a cache file. It also prints a summary line and an exit code.\n\nThe listing below is a labeled example, not production code. It mixes parsing, caching, and printing in one function. That mix is the core extract hazard.\n\n``` python\n# proposal: messy glue, not a live codebase\nimport csv, hashlib, json, os, sys\n\nCACHE = os.environ.get(\"REPORT_CACHE\", \".cache/report.json\")\n\ndef run():\n    os.makedirs(os.path.dirname(CACHE) or \".\", exist_ok=True)\n    rows = list(csv.DictReader(sys.stdin))\n    total = sum(int(r.get(\"amount\") or 0) for r in rows)\n    payload = {\"count\": len(rows), \"total\": total}\n    digest = hashlib.sha256(\n        json.dumps(payload, sort_keys=True).encode()\n    ).hexdigest()[:12]\n    with open(CACHE, \"w\", encoding=\"utf-8\") as f:\n        json.dump({**payload, \"digest\": digest}, f)\n    print(f\"rows={payload['count']} total={payload['total']} digest={digest}\")\n    return 0\n\nif __name__ == \"__main__\":\n    raise SystemExit(run())\n```\n\nDo not split the run function just yet. First pin what the whole process does. Then change one seam only after that pin.\n\nThe harness below is a proposal you can copy. It runs one command inside a temp workspace. It writes a stable JSON ledger for later diffing.\n\n``` python\n# proposal: characterization harness, unexecuted here\nimport hashlib, json, os, shutil, subprocess, tempfile\nfrom pathlib import Path\n\nVOLATILE_ENV = {\"PWD\", \"OLDPWD\", \"SHLVL\", \"SSH_AUTH_SOCK\", \"TERM\"}\n\ndef hash_tree(root: Path) -> dict:\n    out = {}\n    for path in sorted(root.rglob(\"*\")):\n        if path.is_file():\n            rel = str(path.relative_to(root)).replace(\"\\\\\", \"/\")\n            out[rel] = hashlib.sha256(path.read_bytes()).hexdigest()\n    return out\n\ndef capture(cmd, stdin_bytes, extra_env):\n    sandbox = Path(tempfile.mkdtemp(prefix=\"ledger-\"))\n    try:\n        env = {\n            key: value\n            for key, value in os.environ.items()\n            if key not in VOLATILE_ENV\n        }\n        env.update(extra_env)\n        proc = subprocess.run(\n            cmd,\n            input=stdin_bytes,\n            cwd=sandbox,\n            env=env,\n            capture_output=True,\n        )\n        return {\n            \"exit\": proc.returncode,\n            \"stdout\": proc.stdout.decode(\"utf-8\", \"replace\"),\n            \"stderr\": proc.stderr.decode(\"utf-8\", \"replace\"),\n            \"files\": hash_tree(sandbox),\n            \"env_subset\": {key: env.get(key) for key in sorted(extra_env)},\n        }\n    finally:\n        shutil.rmtree(sandbox, ignore_errors=True)\n\ndef write_ledger(path, cases):\n    Path(path).write_text(json.dumps(cases, indent=2, sort_keys=True) + \"\\n\")\n```\n\nAdd a tiny fixture runner next to the harness. Keep the fixtures boring, small, and fully deterministic. Three fixture cases beat one lucky happy path.\n\n``` python\n# proposal: fixture pack for report.py\nimport sys\nfrom pathlib import Path\n\nREPORT = str(Path(\"report.py\").resolve())\nCASES = [\n    {\n        \"name\": \"empty_stdin\",\n        \"stdin\": b\"\",\n        \"env\": {\"REPORT_CACHE\": \"out/report.json\"},\n    },\n    {\n        \"name\": \"two_rows\",\n        \"stdin\": b\"amount\\n10\\n5\\n\",\n        \"env\": {\"REPORT_CACHE\": \"out/report.json\"},\n    },\n    {\n        \"name\": \"missing_amount\",\n        \"stdin\": b\"name\\nalice\\n\",\n        \"env\": {\"REPORT_CACHE\": \"out/report.json\"},\n    },\n]\n\ndef main(out_path):\n    cmd = [\"python\", REPORT]\n    ledgers = {}\n    for case in CASES:\n        ledgers[case[\"name\"]] = capture(cmd, case[\"stdin\"], case[\"env\"])\n    write_ledger(out_path, ledgers)\n\nif __name__ == \"__main__\":\n    main(sys.argv[1] if len(sys.argv) > 1 else \"ledger.before.json\")\n```\n\nStore the before ledger inside the git history. Treat that committed file as a characterization oracle. Diff the ledger file after every candidate extract.\n\nFollow these seven steps in strict order. Skip none of these steps on messy glue.\n\nInventory the process contract in one short note. List stdin, env keys, output files, and exit codes. Do not list internal helper names just yet.\n\nWrite the note as a bullet list in the PR. Name each env key and its default value. Name each file path the process may create.\n\nBuild a fixture pack from real sample inputs. Prefer anonymized production-like rows over synthetic ones. Keep each fixture file under a few kilobytes.\n\nInclude empty stdin, two valid rows, and a missing column. Do not include live hostnames or access tokens. Redact any customer field before the file lands.\n\nRun the harness and commit the before ledger. Confirm the empty, happy, and malformed cases exist. Confirm that file hashes are relative and stable.\n\nCommit harness code with the ledger in one snapshot. That pair is the oracle for later diffs. Do not fold extra formatting into this snapshot.\n\nFreeze the git repo at that ledger commit. Do not format, rename, or extract in the same commit. The ledger commit must contain zero behavior edits.\n\nTag the commit if your team likes movable pointers. A local branch name is enough for a solo run. The point is a restore target after a bad extract.\n\nPropose the smallest extract that preserves the ledger. One function move is enough for this step. Do not relocate caches or change env names.\n\nMove only the cache write into a helper. Leave CSV parsing inside the original run function. Leave stdout formatting inside the original run function.\n\n``` python\n# proposal: smallest extract after the ledger pin\nimport csv, hashlib, json, os, sys\n\nCACHE = os.environ.get(\"REPORT_CACHE\", \".cache/report.json\")\n\ndef write_cache(path, record):\n    os.makedirs(os.path.dirname(path) or \".\", exist_ok=True)\n    with open(path, \"w\", encoding=\"utf-8\") as handle:\n        json.dump(record, handle)\n\ndef run():\n    rows = list(csv.DictReader(sys.stdin))\n    payload = {\n        \"count\": len(rows),\n        \"total\": sum(int(r.get(\"amount\") or 0) for r in rows),\n    }\n    digest = hashlib.sha256(\n        json.dumps(payload, sort_keys=True).encode()\n    ).hexdigest()[:12]\n    write_cache(CACHE, {**payload, \"digest\": digest})\n    print(f\"rows={payload['count']} total={payload['total']} digest={digest}\")\n    return 0\n```\n\nRe-run the same harness into the after ledger. Diff the two JSON files with a structured tool. Any extra JSON key is a failed characterization.\n\n```\npython harness.py ledger.before.json\ngit add ledger.before.json report.py harness.py\ngit commit -m \"test: pin report glue side-effect ledger\"\n\n# smallest extract only, after the pin commit\npython harness.py ledger.after.json\npython - <<'PY'\nimport json\nfrom pathlib import Path\nbefore = json.loads(Path(\"ledger.before.json\").read_text())\nafter = json.loads(Path(\"ledger.after.json\").read_text())\nprint(\"match\" if before == after else \"mismatch\")\nif before != after:\n    for name in sorted(set(before) | set(after)):\n        if before.get(name) != after.get(name):\n            print(\"case\", name)\nPY\n```\n\nKeep the extract only when the ledgers match. Revert the whole extract on the first mismatch. Do not \"fix forward\" inside the same change.\n\nA later commit may fix a real product bug. That fix needs new fixtures and a new pin. Do not hide a bugfix inside a rename.\n\nUse this table before you accept a diff.\n\n| Signal | Keep the extract | Revert the extract |\n|---|---|---|\n| Exit codes | Identical per fixture | Any fixture changed |\n| stdout / stderr | Byte-stable after path rewrite | New warnings or missing lines |\n| File set | Same relative paths | Extra cache or dropped file |\n| File hashes | Same digest per path | Any digest changed |\n| Env contract | Same keys read and written | New required key |\n| Diff size | One symbol moved | Helpers, names, and cache all move |\n\nIf three or more rows say revert, stop. The change is not the smallest safe extract. Split the work and then pin again.\n\nByte-stable stdout still fails when a new file appears. New files are process behavior, not formatting noise. Treat those new files as a hard mismatch.\n\nA coding model can suggest the extract later. The model still cannot invent the characterization oracle. Generate patch candidates only against a committed ledger.\n\nDisclosure: This article was prepared as part of MonkeyCode's product outreach. MonkeyCode offers free model access and a free server option.\n\nUse the server to run the harness on a clean workspace. Use a free model to propose one extract after the pin.\n\nDo not ask the model to invent fixtures from memory. Paste the inventory note and the ledger schema instead. Reject any patch that touches cache paths and helpers together.\n\nThis ledger ignores in-process mocks and thread timing. It will miss behavior that never touches stdout or files. Shared network calls need a different freeze, such as recorded HTTP.\n\nHashing whole trees can hide permission-only file changes. Empty directories may disappear from the file map. Binary files need an explicit allow list on large repos.\n\nFixture packs go stale when product rules change. A matching ledger can still encode a bug. Characterization preserves current behavior, including known bad behavior.\n\nThe harness also assumes a single process per fixture. Child daemons will outlive the sandbox cleanup step. Stop those child jobs before hashing the tree.\n\nDo not use this on greenfield modules with real tests. Do not use this when the contract is a public API suite. Do not use this for security-sensitive parsers without a fuzzer.\n\nSkip it when the glue is a thin wrapper over a stable library. Skip it when you cannot sandbox file writes. Skip it when fixtures would include live secrets.\n\nSkip it when the script must talk to a live billing API. Recorded side effects would then include third-party state. That class of glue needs contract tests, not file hashes.\n\nPin the side-effect ledger before any extract. Extract one seam only after the ledgers exist. Then diff the two JSON ledger files.\n\nThat strict order is the entire safety method. The coding model remains optional in this workflow. The characterization oracle is not optional here.\n\nIf you run this on a free workspace, keep the ledger in git. Leave the model out until the oracle exists.", "url": "https://wpnews.pro/news/snapshot-file-deltas-before-you-extract-glue", "canonical_source": "https://dev.to/hackrs_6393/snapshot-file-deltas-before-you-extract-glue-21a0", "published_at": "2026-09-03 16:37:48+00:00", "updated_at": "2026-09-03 16:56:25.843213+00:00", "lang": "en", "topics": ["developer-tools", "mlops"], "entities": [], "alternates": {"html": "https://wpnews.pro/news/snapshot-file-deltas-before-you-extract-glue", "markdown": "https://wpnews.pro/news/snapshot-file-deltas-before-you-extract-glue.md", "text": "https://wpnews.pro/news/snapshot-file-deltas-before-you-extract-glue.txt", "jsonld": "https://wpnews.pro/news/snapshot-file-deltas-before-you-extract-glue.jsonld"}}