Originally published on hexisteme notes.
Part of how I run background coding tasks is by shelling out to a subscription-gated CLI from a different vendor than my main assistant, one task at a time. No daemon, no shared server — each worker starts, does one job, and exits, and I read back whatever it produced. That pattern itself is fine.
What I got wrong for a while was how I decided whether a worker had actually done anything.
On 2026-08-07, one of these workers — OpenAI's Codex CLI, invoked non-interactively — hit its own usage cap mid-task. The real failure message was:
ERROR: You've hit your usage limit
printed to stderr. Meanwhile:
If a dispatcher only checks $?
, this looks identical to a worker that quietly finished a trivial task and had nothing to say. There's no crash, no nonzero status, no exception to catch anywhere in the normal control flow. The failure is completely real; it's just filed under the wrong file descriptor, and the exit code actively lies about it.
A tool that fails loudly — nonzero exit, a stack trace, a panic:
line — is annoying but honest: an if $? -ne 0
branch catches it whether or not you anticipated the specific failure mode. This is a different shape of problem. The interface contract the orchestration is trusting — exit code as the success/failure signal — stays green, while the actual work product (stdout) is empty. Anyone who wires "exit code equals 0" to "mark the task done, move on" will silently record a quota death as a completed job.
timeout
It gets worse on macOS specifically: there's no timeout
binary by default, so wrapping a worker call in timeout ...
on a machine where it doesn't resolve hands back a shell "command not found" as exit 0 — a second, independent path to the same false-success signal.
I'd already tripped over a version of this once before, with a different vendor's CLI, and filed it away as "that tool is just weird about how it reports quota." Watching the identical shape — real error on stderr, empty stdout, exit 0 — show up in a completely separate CLI from a completely different vendor changed the diagnosis: this isn't a bug in one wrapper, it's how subscription-gated command-line tools tend to communicate "you're out of quota." They treat it as a billing condition rather than a program error, so the message goes to stderr and the process exits cleanly rather than breaking a caller's shell pipeline with a nonzero status.
Once that's the assumption, you stop trusting exit codes from any subscription CLI by default.
The rule I apply to every worker dispatch now is boring and mechanical, which is the point:
timeout
exists on every machine it runs on, that assumption is itself a failure mode to check for — not just the worker's behavior.None of this requires knowing anything vendor-specific ahead of time. It requires treating "exit 0" as one weak signal among several, not the whole verdict.
This isn't specific to LLM CLIs — any pipeline-friendly tool tends to swallow certain failure classes into stderr-plus-exit-0 rather than a hard nonzero exit, because tool authors don't want a quota, rate-limit, or auth condition to break a caller's pipeline. The fix generalizes too: treat the exit code of anything you didn't write as a hint, and check the actual output before marking a step done.
I've also hit the mirror-image bug elsewhere — a wrapper failing while the underlying capability still works (a false negative); this one is the opposite: dead on arrival, but the harness says yes (a false positive).
The same distrust-the-green-light instinct applies at the HTTP layer — I've separately seen a 200 OK with an error payload inside get cached as if it were good data; same root cause, different transport.
More notes at hexisteme.github.io/notes.