Every team running AI agents unattended has a version of this story.
The agent was told to clean up build artifacts. It did — plus a directory
one level too high, with total confidence, at 3 a.m. Or it read .env
into context while "just checking the config." Or a scheduled job ran
twice because the first run looked like it failed, so something retried
it — and two agents wrote to the same state at once.
And then there is the failure that makes no sound at all: the nightly
job that simply stopped running. No error, no alert. You find out three
weeks later, from the missing data.
None of these are model-quality problems. A smarter model would have
made the same mistakes, faster and with better prose. They are
operations problems — and operations problems are not fixed by
instructions.
"Be careful with deletions" in a prompt is a request. It competes with
every other token in the context, it decays over a long session, and it
loses to a plausible-sounding instruction buried in a file the agent was
asked to process.
What actually stops a bad tool call is a mechanism that does not depend
on the model remembering to be careful:
curl | sh..env,
.ssh/, and credential stores are not readable in the first place,
no matter what the task seems to require.
This is not new thinking. It is how every other discipline that runs
machinery unattended already works. Nobody writes "please be careful"
on a factory press and calls it a safety system. There is a guard, a
lockout procedure, and a checklist — because the cost of the bad day is
asymmetric.
Running agents in production, we ended up with the same five layers
every time. None of them is glamorous. All of them earn their keep.
1. Written principles and prohibitions. Not in your head — in the
instruction file the agent actually reads. Absolute prohibitions
(production data, secrets, external sending, money movement), read
scope vs. write scope as separate lists, and a staged permission model:
read-only → drafting → approval → limited autonomy, with promotions
earned by evidence, not granted by optimism.
2. Guardrails (enforcement code). Hooks and permission profiles
that physically stop dangerous calls. In our case, for Claude Code:
a destructive-command guard, a secrets guard, a git/production
confirmation guard, and a session-budget guard — with ready-made
settings profiles for each permission phase. Ours were verified by
live-fire testing on Claude Code 2.1.309, and they ship with their
test suite, because a guard you have not tested against the commands
it claims to stop is a hope, not a guard.
3. Verification. Writes are read back from their destination, or
they are unverified — not "done." And when an operation makes a claim
("this change will fix the overlap problem"), the claim is recorded
in gradable form before the outcome is known, then graded later:
HIT, MISS, HOLD, or INVALID. Editing the prediction after seeing the
result is how operations fool themselves.
4. Always-on discipline. Scheduling that prevents overlap and
decides in advance what a missed run means (skip it, catch up once,
catch up all). State files with a single writer and atomic updates. A run log you monitor for absence — silence is an alarm, not a relief.
5. Runbooks and rollout gates. The first five minutes of an
incident, written before the incident: runaway agent, duplicate
execution, partial failure, silent failure. And a road to production
with gates — Backtest → Shadow → Limited Production → Full
Automation — each with pass criteria and, just as important, demotion
rules for when an automation has to go back a stage.
Pattern-based hooks are defense in depth, not a sandbox. A determined
or sufficiently confused agent can route around string matching; the
pack we built says so in its own documentation, because a guardrail
whose limits you don't know is decoration. The point of the layers is
that no single mistake — one misread instruction, one bad retry, one
silent scheduler — gets to become the incident by itself.
We put the smallest useful pieces into a free sample:
CLAUDE.md skeleton you can copy into a project today,
**Free sample (pay what you want, $0 works):**
[https://rinworklab.gumroad.com/l/ai-agent-operations-safety-pack-free-sample](https://rinworklab.gumroad.com/l/ai-agent-operations-safety-pack-free-sample)
The full AI Agent Operations & Safety Pack turns those samples
into the complete system — the principles, the SKILL.md specification,
the full guardrail suite with permission profiles and its 137-case
test suite, verification formats, always-on operations design,
incident runbooks, and the four-gate rollout method:
**Full pack (Standard $99 / Complete $179, 30-day refund):**
[https://rinworklab.gumroad.com/l/ai-agent-operations-safety-pack](https://rinworklab.gumroad.com/l/ai-agent-operations-safety-pack)
No guarantees — there aren't any, here or anywhere. Just fewer 3 a.m.
incidents, and failures that show up where you can see them.
Written by RIN WORK LAB, from running unattended automations in production and from the manufacturing floor's discipline: safety first, one variable at a time, verify before you proceed.