I run a one-person AI company: Claude Code writes and maintains the code, I make the calls that need a human. Most of what it builds runs unattended — a live strategy bot on a 5-minute scheduler, a weekly content pipeline that drafts and publishes without me reading the draft first (I don't read English well enough to review it myself).
That arrangement worked, until two specific moments where "the agent is competent" and "the agent is safe to leave unattended" turned out to be different properties. Here's what happened both times, and the gate I built after the second one so it can't happen a third way.
I asked the agent to replace a losing trading strategy with a new one, backtested and cleared against a profit-factor gate. It rewrote run.py
, saved the file, and was partway into explaining the change to me.
The bot's Windows Task Scheduler entry runs every 5 minutes, unconditionally, whether or not a human is mid-review. It fired. The new code had a bug: it treated the current day's still-forming price candle as a closed one, and used it to make a real (paper) sell decision on incomplete data.
Caught it fast, pulled the scheduled task, and went looking for how deep the problem went. It was deeper than the one bug:
Three separate holes in the safety net, found only because the first one fired. Fixed all four, verified live, re-enabled the schedule. Lesson kept for next time, in plain language: disable the scheduled task before you let an agent touch the file it drives — not after you find out why that mattered.
Partway through the fixes above, I ran git reset --hard HEAD~1
to undo one bad test commit. --hard
doesn't undo one commit — it discards every uncommitted change in the working tree. That included the security fixes from the incident above, not yet committed. Redid them from scratch.
Nothing sophisticated went wrong here. A destructive git command did exactly what it's documented to do, and an agent moving fast used it without pausing to check what else was sitting uncommitted. The kind of mistake a careful senior engineer makes maybe once a decade. An agent making hundreds of commits a week will make it a lot sooner than that, if nothing checks first.
By the time the content pipeline existed — draft, security-check, publish, no human reads the English first — I'd learned enough not to ship that pipeline without a hard gate in front of the publish step. Three layers, any one of them blocks: a hand-maintained deny-list of known-sensitive strings, a live scan that reads every .env
file and checks whether any current secret value appears verbatim in the draft, and a semantic pass from a second model call asking "does this look safe to publish" with the actual draft in front of it, not a rule list.
First real test, unplanned: the pipeline drafted a post about the week's engineering work. The gate caught three real account numbers in the draft and blocked the publish before it reached git or the blog. One of those numbers wasn't even on the hand-written deny-list — it got caught by a generic "this looks like an account-number-shaped string" pattern, which is exactly the point of having more than one layer. Draft sits in a gitignored folder to this day, never published, cursor never advanced past it.
That's the difference between the first two incidents and the third: the first two, I found out about after something already happened. The third, I found out about because the gate was already there and did its job before anything happened. That gap — noticing after vs. preventing before — is the whole reason this is a post and not just an incident log.
An AI agent that writes good code and an AI agent that's safe to run unattended are not the same claim, and testing the first one tells you very little about the second. Every hole above got found by something actually going wrong first, not by anyone predicting it — the circuit breaker's blind spots only surfaced because the first bug fired; the git-reset danger only got named after it ate real work.
I'm pulling the gate — the deny-list plus live-secret-scan plus semantic-check pattern — into a standalone tool for solo developers and small teams running AI agents with real write access (a live bot, a publish pipeline, a deploy step) without a human reviewing every action in real time. Not a general secrets scanner — there are good funded ones already (GitGuardian, gitleaks). This is specifically for the moment an agent is about to do something irreversible unattended, and nobody's watching that specific second.
If you're running an agent with any kind of unattended write access and have your own version of incident 1 or 2, I'd like to hear it — trying to build this from more than my own two data points.