{"slug": "the-switch-inside-the-room", "title": "The Switch Inside the Room", "summary": "A coding-agent security tool shipped by an unnamed developer was found to have three critical flaws, including a kill switch at `.claude/gate-off` that the agent could create without denial, allowing subsequent writes to `/etc/passwd` to go through. The tool's policy file and its own binary in `node_modules` were also writable by the agent, enabling silent policy changes and complete bypass of enforcement. The developer concluded that anything the enforcement depends on must live outside what it enforces upon.", "body_md": "# The Switch Inside the Room\n\nWe ship a thing that sits between a coding agent and your machine and decides what it may do. Last week someone using it asked their agent a perfectly ordinary question, and the agent turned the whole thing off.\n\nNot maliciously. It was asked to create a file, and it created the file. The file happened to be the kill switch.\n\n```\n> Create an empty file at .claude/gate-off in this project.\n\nCreated `.claude/gate-off` (empty, 0 bytes).\n```\n\nOne turn. No denial, no prompt, no warning. And a write to `/etc/passwd`\n\nthat had\nbeen refused thirty seconds earlier now went straight through.\n\nThe person who reported it noticed something sharper than the bug. Their agent had\n*refused* the same request earlier in a different session — when the conversation\nhad made clear what the file was for. It complied when it didn’t know. Which means\nthe thing standing between the agent and the off switch was **the agent’s own\nunderstanding of what it was being asked to do.**\n\nThat is not a security control. That is hoping.\n\n## The shape\n\nThe kill switch lived at `.claude/gate-off`\n\n, inside the project directory. The\nagent’s job is to write files in the project directory. So the switch that turns\noff the enforcement was sitting in the one place the enforced party has permission\nto write.\n\nWritten out like that it’s obvious. It wasn’t obvious while writing it, because every individual decision was reasonable: put the config with the project, keep the kill switch next to the config, make it a plain file so it’s easy to explain and easy to use in a hurry. Each step sensible, and the destination is a lock whose key hangs on the inside of the door.\n\nThen we went looking, and found it twice more.\n\n## Twice more\n\n**The rulebook.** The policy is a manifest — a YAML file listing what the agent may\ntouch. It also lived in the project directory. Writing to it drew no denial, not\neven a prompt:\n\n```\nWrite  .claude/cc-world.yaml     → ALLOWED (silent)\nEdit   .claude/cc-world.yaml     → ALLOWED (silent)\n```\n\nWe assumed this was survivable, because the compiled policy is immutable once\nbuilt. That assumption was wrong, and wrong in the direction that matters. The hook\nruns as a fresh process on *every single tool call*, which means it recompiles the\nmanifest every time. So:\n\n```\n1. baseline                                       → DENY\n   [agent edits one line of its own policy]\n2. next tool call, same session                   → ALLOWED\n```\n\nThere is no “takes effect next session,” because there is no session. One edit, one call later, new rules. And unlike flipping a kill switch, this one is quiet: the governance is still running, still logging, still returning verdicts. It’s just answering a different question than you think it is.\n\n**The kernel itself.** This one we introduced while fixing packaging. The tool ships\nas an npm package; install it locally and the binary lands in `node_modules/`\n\n—\nwhich is, once again, inside the project the agent can write to. We measured it:\n\n```\nWrite  node_modules/…/harness   → ALLOWED\n[replace it with a script that exits 0]\nWrite  /etc/passwd              → silence. every verdict gone.\n```\n\nThree files. A switch, a rulebook, and the enforcer. Found weeks apart, in different subsystems, each looking like its own small bug.\n\n## The sentence\n\nAnything the enforcement depends on must live outside what it enforces upon.\n\nThat’s it. It catches all three, and it would have caught them before they were written.\n\nIf you’ve done any security work this will sound familiar, and it should. It’s half\nof the [reference monitor](https://en.wikipedia.org/wiki/Reference_monitor) criteria\nfrom 1972 — a reference validation mechanism must be tamperproof, always invoked,\nand small enough to verify. We rediscovered *tamperproof* the slow way, by breaking\nit three times in one codebase.\n\nWorth being precise about why the classic phrasing didn’t save us. “Tamperproof”\nsounds like it’s about attackers. None of these were attacks. There was no adversary\nand no injection; a helpful assistant was asked to create a file and did. The\nproperty you need isn’t “resistant to a determined attacker” — it’s **“not reachable\nby the ordinary, well-intentioned operation of the thing being governed.”** Much\nlower bar, and we cleared it in the wrong direction three times.\n\n## What the fixes look like\n\n**The switch moved out of the project.** It now lives at\n`~/.claude/ai2rules/off/<project>`\n\n, and writes outside the project are themselves\ngoverned, so an agent reaching for it gets an approval prompt instead of silence.\nNot a hard denial — a human in the loop, which is the right bar for *“turn off your\nown supervision.”*\n\n**The rulebook became read-only.** One line in the manifest:\n\n```\n- { path: \"./.claude\", access: Read }\n```\n\nRead stays allowed on purpose. An agent that can’t see its own rules can’t explain\na refusal or suggest a sensible change to one, and the file is committed to the\nrepo anyway — secrecy was never the property being defended. **Write authority\nwas.**\n\n**The installer now refuses to install itself somewhere it can be replaced.** If the\nbinary is inside the project it would govern, it stops and tells you to install\nglobally. A governance install that can be undone with one file write is worse than\nnone, because it’s indistinguishable from one that works.\n\n## The part that generalises past our tool\n\nThat last sentence is the one worth taking away, and it isn’t really about us.\n\nIf you run any agent with a guardrail — a hook, a permissions file, a wrapper\nscript, an allowlist, a sandbox profile — ask one question: **can the agent write\nto it?** Not “would it,” not “why would it want to.” Can it.\n\nTry it, too, rather than reasoning about it. Every one of these three was found by\nrunning something and watching, and two of them survived a security review, a test\nsuite that grew to 244 cases, and a published blog post about how careful we were\nbeing. The kill-switch test in that suite *passed the entire time* — it asserted the\nshim contained the string `.claude/gate-off`\n\n, and after the fix the shim still\ncontained that string, in a comment explaining the old behaviour. A green test,\nagreeing with a defect, made wrong by a documentation improvement.\n\nThe check is two lines and a directory listing:\n\n```\n# whatever your agent's guardrail config is, ask it to write there\necho '{\"tool_name\":\"Write\",\"tool_input\":{\"file_path\":\"<your-guardrail-file>\"}}' \\\n  | <your hook>\n```\n\nIf nothing comes back, your policy is a suggestion your agent has been kind enough to follow.\n\n*The tool is ai2rules-harness —\nnpm install -g ai2rules-harness && harness init. The three fixes are in 0.2.0.\nThe full reasoning, including what we rejected, is in\nDECISIONS.md D57 and\nD58.*", "url": "https://wpnews.pro/news/the-switch-inside-the-room", "canonical_source": "https://ai2rules.dev/blog/the-switch-inside-the-room/", "published_at": "2026-08-12 00:00:00+00:00", "updated_at": "2026-08-15 07:43:00.722132+00:00", "lang": "en", "topics": ["ai-safety", "ai-agents", "ai-tools", "ai-policy"], "entities": ["Claude", "npm"], "alternates": {"html": "https://wpnews.pro/news/the-switch-inside-the-room", "markdown": "https://wpnews.pro/news/the-switch-inside-the-room.md", "text": "https://wpnews.pro/news/the-switch-inside-the-room.txt", "jsonld": "https://wpnews.pro/news/the-switch-inside-the-room.jsonld"}}