# Snapshot File Deltas Before You Extract Glue

> Source: <https://dev.to/hackrs_6393/snapshot-file-deltas-before-you-extract-glue-21a0>
> Published: 2026-09-03 16:37:48+00:00

Do not extract glue until side effects are pinned. Messy modules leak behavior through files, env, and cwd. A function signature is not a characterization test.

AI diffs look clean when tests are missing. Glue code still writes caches, logs, and temp files. Reviewers then ship a rename that changes process behavior.

Cheap code generation does not reduce the characterization cost. Technical debt remains in implicit state, not syntax. You need a ledger before the first extract.

Untested glue often reads hidden config from the process. It may chdir, mkdir, or rewrite a lockfile. Those actions never appear in the public return value.

Return values also miss mkdir side effects on empty input. Missing directories can change later commands in the same job. That is still user-visible behavior, not an internal detail.

A ledger is a frozen record of process behavior. It is not a unit test of internal helpers. It pins observable results for a fixed fixture pack.

Record five observable channels on every fixture run. Capture exit code, stdout, stderr, file deltas, and env keys. Ignore wall-clock time unless the contract requires it.

Hash file contents, not only the file names. Store relative paths instead of absolute sandbox paths. Drop volatile fields like pid, dates, and hostnames.

Normalize stdout by stripping any sandbox path prefix. Keep the newline style identical across all runs. Sort JSON object keys before hashing cache files.

Consider a report glue script with no tests. It reads CSV from stdin and writes a cache file. It also prints a summary line and an exit code.

The listing below is a labeled example, not production code. It mixes parsing, caching, and printing in one function. That mix is the core extract hazard.

``` python
# proposal: messy glue, not a live codebase
import csv, hashlib, json, os, sys

CACHE = os.environ.get("REPORT_CACHE", ".cache/report.json")

def run():
    os.makedirs(os.path.dirname(CACHE) or ".", exist_ok=True)
    rows = list(csv.DictReader(sys.stdin))
    total = sum(int(r.get("amount") or 0) for r in rows)
    payload = {"count": len(rows), "total": total}
    digest = hashlib.sha256(
        json.dumps(payload, sort_keys=True).encode()
    ).hexdigest()[:12]
    with open(CACHE, "w", encoding="utf-8") as f:
        json.dump({**payload, "digest": digest}, f)
    print(f"rows={payload['count']} total={payload['total']} digest={digest}")
    return 0

if __name__ == "__main__":
    raise SystemExit(run())
```

Do not split the run function just yet. First pin what the whole process does. Then change one seam only after that pin.

The harness below is a proposal you can copy. It runs one command inside a temp workspace. It writes a stable JSON ledger for later diffing.

``` python
# proposal: characterization harness, unexecuted here
import hashlib, json, os, shutil, subprocess, tempfile
from pathlib import Path

VOLATILE_ENV = {"PWD", "OLDPWD", "SHLVL", "SSH_AUTH_SOCK", "TERM"}

def hash_tree(root: Path) -> dict:
    out = {}
    for path in sorted(root.rglob("*")):
        if path.is_file():
            rel = str(path.relative_to(root)).replace("\\", "/")
            out[rel] = hashlib.sha256(path.read_bytes()).hexdigest()
    return out

def capture(cmd, stdin_bytes, extra_env):
    sandbox = Path(tempfile.mkdtemp(prefix="ledger-"))
    try:
        env = {
            key: value
            for key, value in os.environ.items()
            if key not in VOLATILE_ENV
        }
        env.update(extra_env)
        proc = subprocess.run(
            cmd,
            input=stdin_bytes,
            cwd=sandbox,
            env=env,
            capture_output=True,
        )
        return {
            "exit": proc.returncode,
            "stdout": proc.stdout.decode("utf-8", "replace"),
            "stderr": proc.stderr.decode("utf-8", "replace"),
            "files": hash_tree(sandbox),
            "env_subset": {key: env.get(key) for key in sorted(extra_env)},
        }
    finally:
        shutil.rmtree(sandbox, ignore_errors=True)

def write_ledger(path, cases):
    Path(path).write_text(json.dumps(cases, indent=2, sort_keys=True) + "\n")
```

Add a tiny fixture runner next to the harness. Keep the fixtures boring, small, and fully deterministic. Three fixture cases beat one lucky happy path.

``` python
# proposal: fixture pack for report.py
import sys
from pathlib import Path

REPORT = str(Path("report.py").resolve())
CASES = [
    {
        "name": "empty_stdin",
        "stdin": b"",
        "env": {"REPORT_CACHE": "out/report.json"},
    },
    {
        "name": "two_rows",
        "stdin": b"amount\n10\n5\n",
        "env": {"REPORT_CACHE": "out/report.json"},
    },
    {
        "name": "missing_amount",
        "stdin": b"name\nalice\n",
        "env": {"REPORT_CACHE": "out/report.json"},
    },
]

def main(out_path):
    cmd = ["python", REPORT]
    ledgers = {}
    for case in CASES:
        ledgers[case["name"]] = capture(cmd, case["stdin"], case["env"])
    write_ledger(out_path, ledgers)

if __name__ == "__main__":
    main(sys.argv[1] if len(sys.argv) > 1 else "ledger.before.json")
```

Store the before ledger inside the git history. Treat that committed file as a characterization oracle. Diff the ledger file after every candidate extract.

Follow these seven steps in strict order. Skip none of these steps on messy glue.

Inventory the process contract in one short note. List stdin, env keys, output files, and exit codes. Do not list internal helper names just yet.

Write the note as a bullet list in the PR. Name each env key and its default value. Name each file path the process may create.

Build a fixture pack from real sample inputs. Prefer anonymized production-like rows over synthetic ones. Keep each fixture file under a few kilobytes.

Include empty stdin, two valid rows, and a missing column. Do not include live hostnames or access tokens. Redact any customer field before the file lands.

Run the harness and commit the before ledger. Confirm the empty, happy, and malformed cases exist. Confirm that file hashes are relative and stable.

Commit harness code with the ledger in one snapshot. That pair is the oracle for later diffs. Do not fold extra formatting into this snapshot.

Freeze the git repo at that ledger commit. Do not format, rename, or extract in the same commit. The ledger commit must contain zero behavior edits.

Tag the commit if your team likes movable pointers. A local branch name is enough for a solo run. The point is a restore target after a bad extract.

Propose the smallest extract that preserves the ledger. One function move is enough for this step. Do not relocate caches or change env names.

Move only the cache write into a helper. Leave CSV parsing inside the original run function. Leave stdout formatting inside the original run function.

``` python
# proposal: smallest extract after the ledger pin
import csv, hashlib, json, os, sys

CACHE = os.environ.get("REPORT_CACHE", ".cache/report.json")

def write_cache(path, record):
    os.makedirs(os.path.dirname(path) or ".", exist_ok=True)
    with open(path, "w", encoding="utf-8") as handle:
        json.dump(record, handle)

def run():
    rows = list(csv.DictReader(sys.stdin))
    payload = {
        "count": len(rows),
        "total": sum(int(r.get("amount") or 0) for r in rows),
    }
    digest = hashlib.sha256(
        json.dumps(payload, sort_keys=True).encode()
    ).hexdigest()[:12]
    write_cache(CACHE, {**payload, "digest": digest})
    print(f"rows={payload['count']} total={payload['total']} digest={digest}")
    return 0
```

Re-run the same harness into the after ledger. Diff the two JSON files with a structured tool. Any extra JSON key is a failed characterization.

```
python harness.py ledger.before.json
git add ledger.before.json report.py harness.py
git commit -m "test: pin report glue side-effect ledger"

# smallest extract only, after the pin commit
python harness.py ledger.after.json
python - <<'PY'
import json
from pathlib import Path
before = json.loads(Path("ledger.before.json").read_text())
after = json.loads(Path("ledger.after.json").read_text())
print("match" if before == after else "mismatch")
if before != after:
    for name in sorted(set(before) | set(after)):
        if before.get(name) != after.get(name):
            print("case", name)
PY
```

Keep the extract only when the ledgers match. Revert the whole extract on the first mismatch. Do not "fix forward" inside the same change.

A later commit may fix a real product bug. That fix needs new fixtures and a new pin. Do not hide a bugfix inside a rename.

Use this table before you accept a diff.

| Signal | Keep the extract | Revert the extract |
|---|---|---|
| Exit codes | Identical per fixture | Any fixture changed |
| stdout / stderr | Byte-stable after path rewrite | New warnings or missing lines |
| File set | Same relative paths | Extra cache or dropped file |
| File hashes | Same digest per path | Any digest changed |
| Env contract | Same keys read and written | New required key |
| Diff size | One symbol moved | Helpers, names, and cache all move |

If three or more rows say revert, stop. The change is not the smallest safe extract. Split the work and then pin again.

Byte-stable stdout still fails when a new file appears. New files are process behavior, not formatting noise. Treat those new files as a hard mismatch.

A coding model can suggest the extract later. The model still cannot invent the characterization oracle. Generate patch candidates only against a committed ledger.

Disclosure: This article was prepared as part of MonkeyCode's product outreach. MonkeyCode offers free model access and a free server option.

Use the server to run the harness on a clean workspace. Use a free model to propose one extract after the pin.

Do not ask the model to invent fixtures from memory. Paste the inventory note and the ledger schema instead. Reject any patch that touches cache paths and helpers together.

This ledger ignores in-process mocks and thread timing. It will miss behavior that never touches stdout or files. Shared network calls need a different freeze, such as recorded HTTP.

Hashing whole trees can hide permission-only file changes. Empty directories may disappear from the file map. Binary files need an explicit allow list on large repos.

Fixture packs go stale when product rules change. A matching ledger can still encode a bug. Characterization preserves current behavior, including known bad behavior.

The harness also assumes a single process per fixture. Child daemons will outlive the sandbox cleanup step. Stop those child jobs before hashing the tree.

Do not use this on greenfield modules with real tests. Do not use this when the contract is a public API suite. Do not use this for security-sensitive parsers without a fuzzer.

Skip it when the glue is a thin wrapper over a stable library. Skip it when you cannot sandbox file writes. Skip it when fixtures would include live secrets.

Skip it when the script must talk to a live billing API. Recorded side effects would then include third-party state. That class of glue needs contract tests, not file hashes.

Pin the side-effect ledger before any extract. Extract one seam only after the ledgers exist. Then diff the two JSON ledger files.

That strict order is the entire safety method. The coding model remains optional in this workflow. The characterization oracle is not optional here.

If you run this on a free workspace, keep the ledger in git. Leave the model out until the oracle exists.