{"slug": "the-file-fence-lab", "title": "The File Fence Lab", "summary": "A developer published a teaching lab that uses a simple \"file fence\" — a two-path allowlist plus a git diff checker — to stop AI coding agents from expanding a one-line timeout fix into an unrequested Redis cache subsystem. The lab has students commit a broken invoice handler, write fence.txt and a checker script, then replay the same \"fix the 504\" prompt with and without the fence, showing that out-of-scope files fail before human review. The author frames the fence as \"a seatbelt for a lab,\" not a sandbox or a replacement for code review.", "body_md": "A Friday review often begins with a polite subject line. The patch claims to fix a 504 on `/invoice`. Then the file list arrives: a new Redis helper, a line in `requirements.txt`, a port in `docker-compose.yml`, and a comment that the timeout was “probably a cache miss.” Nobody asked for a cache. The timeout lived in one function. The model widened the job because nothing in the repo told it where the fence was.\n\nThis lab treats that leak as a teaching object, not a personality debate. AI coding loops are good at local edits and reckless at scope. A file fence is a boring control: a short allowlist, a checker that reads `git diff`, and a golden test that still fails if the timeout is missing. Students can rerun the whole path on a laptop in ninety minutes. The point is not to win an argument about agents. The point is to make an out-of-scope file fail before a human has to notice Redis.\n\nThe fence is a garden gate, not a vault. It stops accidental sprawl. It does not sandbox the model, and it does not replace code review. Treat it as a seatbelt for a lab, then keep the skepticism.\n\nThe session opens with a ten-minute story and a broken invoice handler. The instructor shows the 504, the single function that calls an upstream billing API, and a failing test that expects `timeout=3`. No architecture diagrams. The class can see the leak coming.\n\nMinutes ten through thirty belong to the fence file. Each pair writes `fence.txt` with two paths: the production module and its test. They are told the next prompt will “fix the 504.” They are not told that a typical model will offer a cache. The constraint has to exist before the prompt, or the prompt will invent a subsystem to look busy.\n\nMinutes thirty through sixty are the checker. Students implement a script that runs `git diff --name-only` against `HEAD` and exits nonzero if any path sits outside the fence. The script is allowed to be ugly. It is not allowed to parse unified diffs by hand when Git already knows the names. A green checker on an empty tree is a trap; they must commit the broken handler first so the later patch has a base.\n\nThe last half hour is a replay. The same prompt is applied twice: once with the fence ignored, once with the checker in the loop. The golden test stays in both runs. A patch that adds Redis and still forgets `timeout=3` fails twice, which is the lesson. A patch that only sets the timeout passes the fence and the test. The debrief is short. Students compare file lists, not vibes.\n\nThe sample app is one module and one test. Save this as `app/invoices.py`.\n\n``` python\n# app/invoices.py\nfrom __future__ import annotations\n\nimport json\nfrom urllib.request import Request, urlopen\n\nUPSTREAM = \"https://billing.example.invalid/invoice\"\n\ndef fetch_invoice(invoice_id: str) -> dict:\n    # Lab bug: the call can hang. The fix is a timeout, not a cache.\n    req = Request(UPSTREAM, method=\"GET\")\n    with urlopen(req) as resp:  # missing timeout=\n        return json.loads(resp.read().decode(\"utf-8\"))\n```\n\nSave the test as `tests/test_invoices.py`. It does not hit the network. It inspects the call the handler should have made.\n\n``` python\n# tests/test_invoices.py\nfrom unittest.mock import patch, MagicMock\n\nfrom app.invoices import fetch_invoice\n\ndef test_fetch_invoice_sets_a_timeout():\n    fake = MagicMock()\n    fake.read.return_value = b'{\"id\": \"inv_1\"}'\n    with patch(\"app.invoices.urlopen\", return_value=fake) as mocked:\n        # urlopen is used as a context manager in the handler.\n        mocked.return_value.__enter__.return_value = fake\n        mocked.return_value.__exit__.return_value = False\n        fetch_invoice(\"inv_1\")\n        kwargs = mocked.call_args.kwargs\n        assert kwargs.get(\"timeout\") == 3\n```\n\nThe fence is two lines. Anything else is a teaching failure waiting to happen.\n\n```\n# fence.txt\napp/invoices.py\ntests/test_invoices.py\n```\n\nThe checker is the artifact that makes the lab replayable. It refuses mystery files even when the model writes a confident commit message.\n\n``` python\n# check_fence.py\nfrom __future__ import annotations\n\nimport subprocess\nimport sys\nfrom pathlib import Path\n\nFENCE = Path(\"fence.txt\")\n\ndef tracked_names() -> list[str]:\n    out = subprocess.check_output(\n        [\"git\", \"diff\", \"--name-only\", \"HEAD\"],\n        text=True,\n    )\n    return [line.strip() for line in out.splitlines() if line.strip()]\n\ndef allowed() -> set[str]:\n    lines = FENCE.read_text(encoding=\"utf-8\").splitlines()\n    return {line.strip() for line in lines if line.strip() and not line.startswith(\"#\")}\n\ndef main() -> int:\n    names = tracked_names()\n    if not names:\n        print(\"check_fence: no diff against HEAD; nothing to prove\")\n        return 0\n    extra = [name for name in names if name not in allowed()]\n    if extra:\n        print(\"check_fence: files outside fence.txt\")\n        for name in extra:\n            print(f\"  {name}\")\n        return 1\n    print(\"check_fence: diff stays inside the fence\")\n    return 0\n\nif __name__ == \"__main__\":\n    raise SystemExit(main())\n```\n\nA clean replay starts from a tiny Git repo. The commands below assume a POSIX shell and Python 3.11 or newer. Students should not skip the first commit. Without it, `git diff HEAD` has no fence post to lean on.\n\n```\nmkdir fence-lab && cd fence-lab\ngit init\nmkdir -p app tests\n# paste the three files, plus an empty app/__init__.py\nprintf '' > app/__init__.py\ngit add app tests fence.txt check_fence.py\ngit commit -m \"lab: broken invoice fetch\"\npython -m pytest -q || true\npython check_fence.py\n```\n\nThe first pytest run should fail. The checker should pass, because nothing has been patched yet. That pair of results is the baseline. A lab without a baseline will treat any later green bar as success.\n\nGive the model a prompt that sounds helpful and under-specified: “Production is throwing 504 on invoice fetch. Make it reliable.” Leave the fence file in the tree. Many loops will add a client, a dependency, and a comment about caching. Students apply that patch on a branch, then run the two gates.\n\n```\ngit checkout -b leaky\n# apply the generated patch, then:\npython check_fence.py\npython -m pytest -q\n```\n\nThe checker should fail as soon as `redis_client.py` or `requirements.txt` appears. The test may still fail if the timeout never landed. That double failure is useful. It shows that extra architecture did not buy a passing contract.\n\nReset and try the intended edit. The handler needs one argument, not a new service.\n\n```\n# intended body of fetch_invoice\nreq = Request(UPSTREAM, method=\"GET\")\nwith urlopen(req, timeout=3) as resp:\n    return json.loads(resp.read().decode(\"utf-8\"))\ngit checkout main\ngit checkout -b tight\n# edit only app/invoices.py\npython check_fence.py\npython -m pytest -q\n```\n\nA tight patch prints a calm checker line and a passing test. Students keep both command transcripts. The workshop is not finished until those two transcripts exist. Memory is a poor lab notebook.\n\nIf the class needs a shared runner so every pair hits the same Git HEAD, a small always-on box is enough to host `check_fence.py` as a hook target. Disclosure: This article was prepared as part of MonkeyCode's product outreach. MonkeyCode's free model access and free server option can sit under that shared runner so the lesson stays on the fence file rather than on each student wiring a personal key. The product is scaffolding here, not the subject of the test.\n\nThe checker reads names, not semantics. A model can still replace `fetch_invoice` with a stub that returns `{\"ok\": true}` and stay inside the fence. That is why the golden test exists. A fence without a contract is a dress code. Anyone can wear a tie and still skip the timeout.\n\nUntracked files are another hole. `git diff --name-only HEAD` will not see a brand-new Redis module until it is staged or committed, depending on how the class applies patches. Instructors should require `git add -A` before the checker, or extend the script to include `git ls-files --others --exclude-standard`. The lab should name that hole out loud. Hidden files are how “tiny helpers” sneak past a gate that only watches tracked paths.\n\nThe fence is also not an authorization system. It does not stop a prompt from reading secrets in the working tree. It does not replace branch protection. Teams that need isolation still need containers, policy, and a human. This exercise teaches scope discipline for generated diffs. It does not teach production security.\n\nSkip the lab when the change is an incident hotfix under a clock, when the repo has no tests, or when the model is being asked to invent an API that does not exist yet. A fence around a blank folder only trains people to rubber-stamp empty allowlists. Skip it for binary assets and generated lockfiles unless the instructor writes those paths down on purpose. A surprise `package-lock.json` rewrite is a real review event; hiding it behind a two-line fence teaches the wrong reflex.\n\nCurrent public threads keep circling the same fatigue: loops that look like agents, patches that look like products, and developers who cannot tell a fix from a side quest. None of that requires a new slogan. It requires a file the checker can read and a test that still cares about `timeout=3`. After ninety minutes the room should have two transcripts, one leaky and one tight. That pair travels better than a take-home opinion about whether the model is “better at coding.” The fence either held or it did not. The test either saw a timeout or it did not. Everything else can wait for the next lab.", "url": "https://wpnews.pro/news/the-file-fence-lab", "canonical_source": "https://dev.to/applab_8831/the-file-fence-lab-4kh4", "published_at": "2026-09-11 15:26:25+00:00", "updated_at": "2026-09-11 15:43:50.139571+00:00", "lang": "en", "topics": ["ai-agents", "ai-tools", "developer-tools", "mlops"], "entities": ["Redis", "Git", "Python", "docker-compose"], "alternates": {"html": "https://wpnews.pro/news/the-file-fence-lab", "markdown": "https://wpnews.pro/news/the-file-fence-lab.md", "text": "https://wpnews.pro/news/the-file-fence-lab.txt", "jsonld": "https://wpnews.pro/news/the-file-fence-lab.jsonld"}}