sideeye kills a program between its state-changing operations and asks what survived: GitHub - yottayoshida/sideeye: A deterministic skeptic for the coding loop: breaks your software's world and brings back the smallest reproducible counterexample. Sideeye doesn't believe it. · GitHub
I have been building small tools for working alongside AI coding agents. Looking back at them I noticed they share a habit: the decision is made by code rather than by a model, and there is always some command you can run to check what I claim. At some point it occurred to me that the habit was mine rather than the tools’. I am a deterministic skeptic and somewhat paranoid about it, so I decided to make that the product. Running it as a CI job seemed like the funny part.
What it looks at is crash consistency: what a program leaves on disk when it dies partway through. Ordinary tests only check the world where the process ran to completion.
It is at v0.13.0 and does not meet its own v1.0 bar. Five of the six entry criteria hold and the first one does not. Until this week nothing I had found was novel and automatically discovered and provenance-clean at once. Something now is, and I still have to decide whether I think the bug is real, fix it, and get a regression case running in CI. The search part works.
Here is what it does. If a program unlinks the old file and dies before renaming the replacement into place, is the invariant you declared still true after a restart? You write the invariant and sideeye invents the failures: one crash world per operation, the state directory restored each time, and the earliest failing crash point handed back as a replayable case. Inputs are never varied; the variable is where the process dies.
The crash it models is a process crash, SIGKILL. The OS stays up, every completed write survives, and nothing is reordered. Power loss and torn writes are outside it, so a PASS says nothing about those. I picked that model because the bugs that fall out of it are bugs in the order and atomicity of the state changes the program itself issues, and finding them needs no filesystem simulation.
ALICE (OSDI '14) did this kind of exploration first, and Pathfinder (OOPSLA 2025) recently cut the state space by exploiting correlation between crash states, finding bugs in eight production-ready systems. What I wanted to know was whether this could be made small and boring enough to run against ordinary CLI tools during development, rather than living as a research setup. sideeye is my attempt at that.
It has produced replay-confirmed counterexamples against six real tools: timewarrior, topydo, GNU Stow, calcurse, devtodo and himalaya. Several are reported upstream. The himalaya one is the easiest to describe: maildir messages copy
, killed after the destination is opened and before the content is written, leaves a message file at its final path with nothing in it, which the tool then lists as an ordinary envelope.
Verdicts are PASS, FAIL and UNKNOWN, decided by code with no model in the path. A target sideeye cannot fully observe comes back UNKNOWN with the name of the detector that refused, and UNKNOWN is never exit 0. sideeye demo
compiles a small planted-bug tool and prints a real FAIL report in about two seconds, which doubles as a smoke test of the binary and shim pair.
Statically linked targets, threads and state-changing raw syscalls are refused, and state has to live in one declared directory. Process-boundary verification runs through strace, so it is Linux only. macOS has no unprivileged oracle, which #181 measured: DTrace’s syscall provider matches no probes under SIP even as root, and dtruss
runs the target, exits 0, and captures no syscalls. fs_usage
does produce a usable account but needs root, which I cannot demand of a default install. A macOS PASS therefore requires --allow-unverified
, and the report records that it is the weaker claim.
Why Zig. What ships is a CLI binary plus a shared library injected into the target (LD_PRELOAD
on Linux, DYLD_INSERT_LIBRARIES
on macOS). The shim runs inside somebody else’s process, so I kept the heap, std’s I/O layer, locks and third-party dependencies out of it. Zig let me write the C-ABI shared library and the CLI in one language, shipping as one executable and one library.
I would like to hear where it refuses on your own tools. Each refusal tells me about a case I have not covered yet.
Built and tested against 0.16.0, which is what CI installs. build.zig.zon
declares 0.16.0 as the minimum.
I used Claude extensively while building sideeye, including for implementation and debugging. PASS, FAIL and UNKNOWN are produced by deterministic code, and no language model is involved in deciding a verdict.
I am a Japanese speaker and used an LLM to translate this post into English, so please forgive anything that reads oddly.