Consider this scene. It is a composite, not a personal war story.
An agent ran overnight on a leftover prompt. Morning git status
showed fourteen files. Two of them implemented the requested endpoint. The rest were a new logger, a renamed helper, a rewritten Dockerfile, and a README that now contradicted the tests. Generation cost was close to zero. The next four hours were not.
That gap is the actual product problem. When a patch is cheap to produce, the expensive work is routing: keep, quarantine, or rewrite. Skip the routing and the cheap code becomes expensive debt with extra files attached.
This article is a glossary, a routing tree, and a worked example at each leaf. The artifact is a small classifier plus a quarantine command sequence. Treat the code as a proposal unless you run it on your own repo.
Free-model loops optimize for “a diff appeared.” Reviewers optimize for “this diff is safe to merge.” Those are different objective functions. A green unit test on a helper you did not ask for is not evidence that the architecture still holds.
Cheap generation also changes failure shape. The common failure is no longer “the model wrote nothing.” It is silent expansion: extra modules, extra dependencies, extra comments that drift from the contract. Routing has to detect that shape before anyone debates style.
Use these terms as they are defined here. Nearby words in vendor blogs do not override them.
Walk the questions in order. Do not skip to a leaf because the diff “looks small.”
Step 1 — Is there an oracle that already fails, or an oracle you can add in under fifteen minutes?
Step 2 — Is surface area bounded?
Bound means: requested paths only, or requested paths plus test files. A hard cap helps. A working default is “three production files and their tests.”
Step 3 — Does the patch need network, secrets, or write access outside a temp directory?
.env
reads): go to Step 4 — Can an isolated process execute the oracle?
The tree is deliberately biased toward discard and rewrite. Cheap generation makes “try it locally” the risky default, not the brave one.
The script below does not prove safety. It only encodes the tree’s cheap heuristics so a human does not re-litigate them every morning. Label: proposal, unexecuted on your tree until you run it.
#!/usr/bin/env python3
"""classify_patch.py — proposal heuristic, not a security scanner."""
from __future__ import annotations
import subprocess
import sys
from pathlib import Path
ALLOWED_PREFIXES = ("src/", "lib/", "tests/", "test/")
ARCH_HINTS = ("auth", "middleware", "migration", "dockerfile", "compose", ".github/")
SECRET_HINTS = ("os.environ", "getenv(", "api_key", "BEGIN ", ".env")
MAX_PROD_FILES = 3
def git_names(diff_range: str) -> list[str]:
out = subprocess.check_output(
["git", "diff", "--name-only", diff_range], text=True
)
return [line.strip() for line in out.splitlines() if line.strip()]
def patch_text(diff_range: str) -> str:
return subprocess.check_output(["git", "diff", diff_range], text=True)
def classify(diff_range: str, requested: set[str]) -> str:
names = git_names(diff_range)
body = patch_text(diff_range).lower()
prod = [n for n in names if not Path(n).parts[0].startswith("test")]
extra = [n for n in names if n not in requested and not n.startswith("test")]
if any(h in n.lower() for n in names for h in ARCH_HINTS):
return "LEAF_D_REWRITE_architecture_touch"
if any(h in body for h in SECRET_HINTS):
return "LEAF_A_DISCARD_secret_or_env_touch"
if extra or len(prod) > MAX_PROD_FILES:
return "LEAF_A_DISCARD_silent_expansion"
if not names:
return "LEAF_A_DISCARD_empty"
if all(n.startswith(ALLOWED_PREFIXES) for n in names) and len(prod) <= 2:
return "LEAF_C_LOCAL_allowlist"
return "LEAF_B_QUARANTINE"
if __name__ == "__main__":
if len(sys.argv) < 3:
print("usage: classify_patch.py <diff-range> <requested-file> [more files]")
sys.exit(2)
decision = classify(sys.argv[1], set(sys.argv[2:]))
print(decision)
Run it against a generated branch, not against main
:
git fetch origin
git checkout -B agent/try-1 origin/agent/try-1
python3 classify_patch.py main...HEAD src/billing/quote.py tests/test_quote.py
The printed leaf is a starting label. Override it when you have information the script cannot see, such as a compliance boundary or a frozen schema.
Scene. Prompt: “Add a quote_total(items)
helper.” Diff: quote.py
, logger.py
, utils/retry.py
, and a new requirements
pin on a metrics SDK.
Why this leaf. Surface area exploded. The extra files are not tests. The metrics SDK implies network. Step 2 and Step 3 both fail.
Worked action.
Task: add quote_total(items) in src/billing/quote.py only.
Do not create files. Do not edit requirements or logging.
Oracle: tests/test_quote.py::test_quote_total_cents must pass.
If the oracle needs a test change, edit that test file only.
What “done” looks like. A new diff with one production file, or a decision to write the helper by hand because the review budget is already spent.
Scene. Prompt: “Parse CSV invoices in src/invoices/parse.py
.” Diff: that file plus tests/test_parse.py
. No auth, no Docker, no env reads. You do not want that parser executing against files in your home directory.
Why this leaf. Bounded surface, no secret touch, and an oracle exists. Isolation is the remaining requirement.
Worked action. Copy the branch into a throwaway directory or machine. Feed only fixture files. Run the oracle. Throw the machine state away.
mkdir -p /tmp/quarantine && cd /tmp/quarantine
git clone --depth 1 --branch agent/try-1 /path/to/local/mirror invoices
cd invoices
python -m venv .venv && . .venv/bin/activate
pip install -e '.[test]'
pytest tests/test_parse.py -q --fixtures-per-test
If the oracle needs a CSV, mount a fixture directory that contains no customer data. If the model added requests.get
, the run still belongs on Leaf A, even if pytest is green: the classifier should have caught it, and the quarantine host should have no egress if you can help it.
A quarantine host can be a local container. It can also be a spare server that never sees production credentials. MonkeyCode is relevant on this leaf only: it currently offers free model access and a free server option, which is one way to keep generation and first-oracle runs off your workstation. Disclosure: This article was prepared as part of MonkeyCode's product outreach. Treat both the models and the server as capacity that can change; verify current terms on the project before you plan around them. Do not place secrets, customer fixtures, or deploy keys on a shared free server.
What “done” looks like. Oracle green on the isolated copy, plus a human skim of the two files before allowlist apply onto a real branch.
Scene. Prompt: “Extract cents(amount: str) -> int
.” Diff: src/money.py
and tests/test_money.py
. Pure functions. No I/O. Classifier prints LEAF_C_LOCAL_allowlist
.
Why this leaf. Isolation would still be nicer, but the blast radius is a two-file pure change and the oracle is local unit tests you already trust.
Worked action.
git checkout main
git checkout agent/try-1 -- src/money.py tests/test_money.py
git diff --cached --stat
pytest tests/test_money.py -q
Stop if --stat
shows any other path. Stop if the test file grew assertions about logging, time, or HTTP. Those are expansion in disguise.
What “done” looks like. Two allowlisted files, oracle green, no other staged paths.
Scene. Prompt: “Make login less flaky.” Diff: auth/middleware.py
, a session store swap, and a Dockerfile change to add Redis.
Why this leaf. Architecture-touching. Step 3’s “real network need” is present, and there is no dedicated staging path in the scene. Quarantine cannot validate session semantics with unit tests alone. Local apply on a developer laptop is how flaky login becomes an outage.
Worked action.
def test_login_failure_body_stable(client):
res = client.post("/login", json={"user": "x", "password": "bad"})
assert res.status_code == 401
assert set(res.json().keys()) == {"error", "code"}
What “done” looks like. A smaller patch that re-enters Leaf B or C, or a human-written change. Not a fourteen-file “login fix.”
The tree does not detect vulnerabilities, license issues, or subtle numeric drift. The classifier is string heuristics. It will miss a polished secret read and it will over-flag a comment that mentions .env
.
Quarantine is not staging. A green oracle on a throwaway host does not mean production behavior. Free model access and a free server do not add an oracle; they only move generation and first execution off your laptop. If you have no contract tests, you do not have Leaf B. You have a remote place to watch the same untested code fail.
Do not use this approach when the repo holds regulated data, when the agent needs production-like secrets, or when the change is a public contract you cannot freeze in a test. Do not use it as a substitute for architecture review on auth, payments, or migrations. Teams that cannot add a fifteen-minute oracle should not scale cheap generation; they should shrink the task.
The routing bias toward discard will feel slow compared with “accept all files.” That slowness is the point. When patches are cheap, the scarce resource is attention. Spend it on oracles and surface-area caps, not on reading surprise Dockerfiles at 9:12 a.m.
If Leaf B is the leaf you keep landing on, verify whether an isolated server is actually isolated, then run the oracle there. MonkeyCode’s free model access and free server option are one place to try that isolation; confirm current availability before you schedule work against them.