# My AI Agents Ship Code While I Sleep. Nobody Reviews It > Source: > Published: 2026-08-18 23:06:19+00:00 [All posts](/blog) # My AI Agents Ship Code While I Sleep. Nobody Reviews It. My AI agents ship code overnight, unreviewed: autonomy gates, verification they can't fake, and graph engineering's building blocks done in plain markdown. *I plan during the day, agents build overnight, and I review when I wake up. This is the system that decides what ships without me.* My morning starts with the board and last night's diffs. While I slept, an AI agent picked ticket `SET-175` off the queue. Four cron jobs were running the same copy-pasted database scan. The agent pulled it into one shared helper, wrote six tests for it, ran the full 2,282-test suite, deployed to dev, and marked the ticket `done` . It even noticed a second ticket describing the same duplication and closed it. **Nobody reviewed any of that before it went out.** I'm the review, the next morning: the diff, the activity log, the board. The same agent left a different ticket unfinished. `SET-184` flags appraisal-gap risk on pending deals using a property-data API. The code was built, tested, and deployed, and it shuts itself off safely if the API fails. It still didn't go live, because turning it on means paying for a data subscription, and no test can tell you if that's worth the money. That call waited for me. **A ticket field called autonomy decided which one could finish without me.** The rest of this post is that field and the system around it. I also think most of the autonomy debate is about the wrong thing. ## Plan all day, build overnight, review in the morning The routine is simple: **Daytime is planning.** Turning ideas into specs, writing acceptance criteria, making the calls agents will need before they need them.**Nights are loops.** A background agent picks the top buildable ticket, builds it end to end, and either finishes or parks it with a question.**Mornings are code review.** Code moves the same way every day. Agents branch, merge to dev, and dev deploys to the dev Amplify site plus the backend. Once a day I review the PR from dev into prod. **That daily PR is the only way anything reaches prod.** #### How code moves Agents get everything up to dev. Prod goes through one PR a day, and I'm the reviewer. Here's the ticket from this morning: **Every call is made in the spec before an agent touches it.** This one even says why it qualified for `auto` : no product decision, and every requirement can be checked by a test. Planning at night and reviewing in the morning is common practice now. 1 Plenty of people run this schedule and wake up to slop. **The difference is what the agent is allowed to finish.** I main Claude Code as my coding agent. Every loop in this post is one of its sessions. The loops themselves are skills: instruction files in the repo, run like commands. I tweak them every time one goes wrong. #### My toolbelt My skills: instruction files in the repo, invoked like commands. Real names from my two repos. ## Why I don't code-review every AI commit The debate right now is about how much to trust the agent. One extreme is YOLO mode: permissions off, guardrails off, agent runs free. 2 Agents are cheap, so the pitch has legs; ship fast, revert whatever breaks. But a revert doesn't un-ship bad data or win back a client's trust. The other extreme reviews every diff, which works until the agent produces more than you can read. Most people sit at the careful end. Only 8% are comfortable with full agent autonomy. 3 Engineers who use AI on most of their work say they can fully hand off almost none of it. The tools split the difference with classifiers that auto-approve safe-looking actions and flag the rest. [4](#ref-4)I think both camps are asking the wrong question. **The right question is: what work can a test prove?** My planning lives in a folder: one markdown file per ticket. Agents edit the files directly. A small local server renders them as a Kanban board for me. Every ticket has one field that decides everything: `autonomy: auto | needs-input | blocked` . A missing field counts as `needs-input` , so **nothing builds by default**. `auto` means an agent can take the ticket all the way to `done` with no human involved. `needs-input` means the ticket waits for me no matter how green its tests are. The question is simple: **can a test prove this correct, or does it need human judgment?** `SET-175` was a pure refactor with tests that fail if the behavior changes. Provable, so it shipped itself. `SET-184` 's code was just as provable and got built; the money call wasn't, so it waited. #### SET-175 vs SET-184 Both tickets got built and verified; the autonomy field decided which one could ship itself. Extract shared scanActiveDeals(statuses) cron helper Pure refactor. Zero behavior change. Six new tests that fail if the scan behavior changes. Flag appraisal-gap risk with AVM + comps on every pending deal Built, tested, deployed. Shuts itself off safely if the property-data API fails. That's the whole idea. **I make the decisions a test can't settle**: product direction, pricing, brand voice, legal text, security policy, client data, anything irreversible. To be clear: **unreviewed means unreviewed to dev**. Agents deploy to a dev environment on dev data. The prod branch is out of their reach. `shipped` only moves when I run the cut myself. Client data sits behind the same gates as pricing and copy. The blast radius of a bad overnight build is a broken dev URL. Here's the full system as a map: #### My board, drawn as a graph Agents and one human as nodes; lanes, promotion, and escalation as edges. Purple edges are provable and run alone; amber edges wait for a person. If you've used LangGraph, this looks familiar. The practice is called graph engineering: wire agent steps into a graph like this one. 5 Every serious framework landed on the same building blocks this year: typed state, conditional routing, checkpoints, interrupt gates. My board has all four as markdown and rules. Frontmatter is the typed state. [6](#ref-6)`autonomy` is the routing. `needs-input` is the interrupt. The activity log is the checkpoint.What I give up is enforcement. A graph runtime can block an illegal move and tell you exactly which step failed. My rules only hold because the model follows them and I catch drift in the morning. What I get back: I can change a rule with a commit, and nothing sits between me and the model. The real difference is who picks the path, the agent or you. 5 A graph locks the route in up front. I let the agent pick its own route and put the effort into checks instead. Anthropic's own agent guide says the same thing: simple, composable patterns over frameworks. Even the graph teams are moving this way: the 2026 trend in production is adding human approval gates before risky actions. [7](#ref-7) [6](#ref-6)## Verification the agent can't fake Unreviewed shipping only works if the model can't declare `done` by itself. Every `auto` ticket has to clear checks that live outside the model: #### My verification ladder Every auto ticket climbs all six rungs. The two starred ones are where 'the AI finished it' claims usually fall apart. Failing vitest tests first, for every acceptance criterion: happy, null/empty, invalid, and the authorization boundary. A Playwright spec drives the real UI via mock-auth role switching. A user-facing change with no spec is not done. react-router typegen + tsc, clean on the touched surface and everything that consumes it. Push the dev branch. Local edits change nothing a live check can see until this happens. Deploy job SUCCEED, dev URL renders, Playwright green against that live URL, error monitor clean on the exercised path. All four, every time. docs/ and CLAUDE.md updated wherever behavior or a contract changed. Shipping isn't done until the docs are. A missing rung means the ticket isn't done, no matter how confident the agent's activity note sounds. Here's an agent finishing `SET-308` , straight from the ticket's activity log (trimmed): ``` 2026-07-03 [claude] DONE (autopilot, dev-verified). ... ADDED the mandated class-retiring guard packages/api/infra-guards.test.ts ... Verification ladder cleared: infra-guards (5, incl. negative control) + full api suite 2718 pass; api+infra typecheck green; deployed SettleStack-Dev (cdk 135s); doc-intelligence E2E PASS vs deployed dev; AWS-log live confirmation — processed a doc on dev, settle-doc-process-dev finalized clean with ZERO ses:SendEmail AccessDenied in the window. Docs synced. Commits 2552b89 + 4b8e352. ``` Most setups skip the live-verify step because **their agent can't see past localhost**. Mine can. It reads the build log when a deploy fails, and checks deploy status before saying it deployed. It tails CloudWatch for new errors on the surface it touched. Playwright runs against the deployed dev URL, not a local server, so the build it tested is the build users hit. When a deploy 500s at 2am, the agent pulls the build log and fixes what it finds. The ticket's activity line says exactly what was checked, including the gaps. #### What my agents touch Every surface my agents can read or drive. The verification row is the part the model can't fake. The harness, code, and planning AWS Verification If you've run Playwright, you're wondering about flaky tests. The rule: re-run the failing spec by itself. If the change caused the failure, fix it or abort the build. If it was already failing before, it goes on a maintenance ticket instead of blocking the queue. **Flaky or not, the check still has to pass.** This is why vibe-coded prototypes fall apart in production: nothing ever tested them outside localhost.[8](#ref-8) The numbers so far, across both projects: 410 tickets, 335 done, 123 shipped to prod. No ticket has ever been reopened after `done` , though the board only knows about the problems my morning reviews caught. Failures happen earlier: `SET-212` failed its E2E check, so the agent reverted its own code and left a question instead of marking it done. Settle's API suite alone is over 2,700 tests. ## AI code that reads like I wrote it The standard criticism of vibe coding is **orphan code**: it works, nobody knows why, and it gets worse with every prompt. 9 The fix is to box the agent in until its output looks like your code. Four rules do it: **Tests first.** TDD, agent-style: a failing test for each acceptance criterion, including the null, invalid, and authorization cases.**One conventions file** the agent reads every session, so patterns and naming stay consistent across months of builds.**Docs in the same commit.** If behavior changes, the docs change with it.**An append-only activity log** on every ticket: what was done, what was verified, what got skipped and why. That's what makes the morning review fast: **the diff reads like I wrote it**. When something needs hand-editing, I open the file and edit it. No digging required. This is also the job context-engineering stacks sell: retrieval pipelines, memory layers. The repo already does it with the conventions file, the docs, and the activity log. The cost is updating the docs on every build. The payoff: the context lives next to the code, so it can't drift. **The model follows whatever patterns already live in your repo**, even without skills or instruction files. React Router gives you two ways to do routes: flat file-based routes, or routes declared manually in config. We wanted flat routes. So every route the AI has added since is a flat route. It just copies what's there. Ship clean patterns and the agent multiplies them. Ship slop and it multiplies that. **Reading the code is still the best success metric there is.** Everyone's trying to engineer past that with full autonomy. Maybe that gets solved in five or ten years. But if it doesn't, the people who kept reading their code will be five or ten years ahead. Anyone who works differently, congrats. At the end of the day I know what went into my repo, I can fix it, and I can debug it. ## Use the stack you knew before AI Every service in that touch map is something I ran before agents existed. React Router, Amplify, Lambda, DynamoDB, CloudWatch, Clerk: I picked them, deployed them, and debugged them by hand. **I built the workflow out of tools I already knew**, on purpose. That matters for two reasons. The toolbelt skills are my own debugging habits written down. The agent knows where the logs are because I knew where the logs are. And the morning review works because I can read everything the agent produced. The framework and the infra fail in ways I've seen before. The agent suggests new libraries and services constantly. The answer is no. Shipping something I don't understand and have never run is tech debt waiting to fuck me later. When a build needs something new, that's a gate like any other: I learn it first, or it waits. ## Clean code still matters We don't do early abstractions either; YAGNI. The repo stays boring on purpose, because **boring is what the model copies best**. Some people say clean code is a thing of the past now that AI writes it. I completely disagree. They've clearly never shipped something that breaks, or something with users. A landing page doesn't need the same checks as an app with multiple users, moving pieces, and AI workflows on top. It always depends. ## How we document Every project documents differently. Here's ours, from [Settle](/blog/building-settle-with-ai): #### How we document Built for the agent that reads it, updated in the same commit as the change. The ADRs are thirteen and counting: why Bedrock over OpenAI, why DynamoDB single-table, why deterministic over AI. Every big decision gets one. One note on comments. They point the model at things: where the docs live, why something is the way it is. That saves [tokens](/blog/what-a-token-really-costs) too. The agent reads a pointer instead of working it out again. Yeah, they can get ugly. Doesn't matter. **The repo is built for agents first**, and the agent is the one in here all day. ## One build at a time My builds run one at a time; one driver per checkout. The swarm pitch is parallel coverage: many agents at once, and a fresh-eyes reviewer catching what the author stopped seeing. Most of what I see is overhead: twenty subagents fan out and come back agreeing with each other, because the same model reading the same context just agrees with itself. Cognition, which runs some of the largest agent fleets, found the same thing with writers: parallel agents make small judgment calls that clash, so the setups that work keep writes single-threaded. 10 My tradeoff: one build a night, and in return no merge referee and no clashing calls. A test suite, a deploy, and a browser give me independent checks; another agent mostly gives me agreement. Git worktrees on Windows have bitten me twice anyway, while the parallel-agent crowd treats worktree-per-task as table stakes. [11](#ref-11)## What I'd delete from my own setup My setup grows bloat too. My written rules drifted: at one point the prod-deploy rule lived in three files with three different answers. The fix was an afternoon of merging them back into one. After all that orchestration reading I wrote an enforcement wishlist for my own board: typed handoffs, executable lane moves, atomic id allocation. Then I checked it against Settle, which runs the same board. Almost none of it was needed: **381 tickets, zero illegal lane moves**. I never built the referee. ## Full autonomy is the wrong destination The usual take is that full autonomy arrives once models stop making mistakes. That's half right: **there are two limits, and only one of them moves**. The limit that moves is what a test can prove. Better models write better tests, pass stricter checks, handle bigger tickets. My `auto` list has grown all year, one way: When an agent hits a decision it can't make, it has to do two things: **name the one decision I need to make, and propose a rule that answers every future question like it**. My answer gets written down. The next agent reads it instead of asking me. `SET-171` is what that looks like. The ticket demanded a browser E2E test for an auth boundary that's only reachable through an LLM over a WebSocket, which would be slow, flaky, and cost money on every CI run. The agent stopped and proposed the rule instead: backend auth boundaries get deterministic integration tests, not browser E2E. I agreed once, the rule went into the janitor's instructions, and no agent has asked since. #### Three questions I only had to answer once board/attention-ledger.md — real entries. Each interruption became a permanent rule instead of a standing invitation to ask again. "Should extracting or deduping SEO/social meta tags need my sign-off?" Rule: A pure 1:1 refactor that changes no emitted tag → auto. Adding or changing a tag is a brand/SEO surface → still mine. "Can an agent consolidate a repeated access-control check on its own?" Rule: Auto, but only using the canonical guard, covering both read and write paths, and shipping a test that proves a non-owner gets denied. "Does adding a database table to the mock-first backend need a human?" Rule: No — that's the normal build pattern, auto. Running a migration against real data is the part that's always mine. Each entry answers a question once instead of every week. That's why this scales: #### Interrupts that learn Cumulative human interrupts as the same classes of question keep recurring. The mechanism at the real ledger's scale: eight AL classes. Each class interrupts once; every recurrence self-serves against the committed rule. **The other limit doesn't move**: what's a fair price, whether a claim is honest, whether copy sounds like the client, whether a migration is worth the risk. A perfect model doesn't turn those into test problems. They stay a person's call. Three things make this work: one human who reads the board every morning, frontier-quality models, and the dev-only blast radius above. It's a solo setup today. With teammates it would need push notifications and per-person gates before I'd trust it. ## Steal the minimal version You don't need my whole setup, and I'm not sharing the kit itself. It's my working gear. But the high-level version is four things: - A folder of markdown tickets. - An `autonomy` field on each one, default needs-input. `done` means deployed, plus one check against the live thing.- A rules file: answer a question once, write it down, the agent reads it next time. Everything else grew from those four after something broke without them. It all comes down to one question: **can a test prove the output, or does a person have to?** The `autonomy` field holds the answer; the ledger moves that line one answered question at a time. More on why this is good for the profession: [AI expands software engineering rather than replacing it](/blog/ai-renaissance-software-engineering). ## References **[1]** [Ship Code While You Sleep: The Overnight Agent Workflow](https://www.developersdigest.tech/blog/overnight-agents-workflow) — Developers Digest, 2026. **[2]** [Claude Code YOLO Mode: Flag, Risks & Safe Setup](https://www.codeagentswarm.com/en/guides/claude-code-yolo-mode-explained) — CodeAgentSwarm, 2026. **[3]** [Agentic AI has big trust issues](https://www.cio.com/article/4087765/agentic-ai-has-big-trust-issues.html) — CIO, 2026. **[4]** [What the 2026 Agentic Coding Trends Report Means for Cybersecurity](https://cloudsecurityguy.substack.com/p/what-the-2026-agentic-coding-trends) — Cloud Security Guy, 2026. **[5]** [Graph Engineering Guide (2026)](https://www.aibuilderclub.com/blog/graph-engineering-guide-2026) — AI Builder Club, 2026. **[6]** [Graph-Based Agent Workflow Orchestration in Production: The 2026 Landscape](https://zylos.ai/research/2026-04-14-graph-based-agent-workflow-orchestration-production/) — Zylos Research, 2026. **[7]** [Building Effective AI Agents](https://www.anthropic.com/engineering/building-effective-agents) — Anthropic, 2024. **[8]** [Limitations of Vibe Coding Tools in 2026](https://www.builder.io/m/explainers/vibe-coding-limitations) — Builder.io, 2026. **[9]** [Vibe Coding Reality Check: Why It's a Technical Debt Nightmare](https://digitalbiztalk.com/article/vibe-coding-reality-check-why-it-s-a-technical-debt-nightmare) — Digital Biz Talk, 2026. **[10]** [Multi-Agents: What's Actually Working](https://cognition.com/blog/multi-agents-working) — Cognition, 2026. **[11]** [Git Worktrees + Claude Code: The 2026 Playbook for Running Parallel Agents](https://www.developersdigest.tech/blog/git-worktrees-claude-code-parallel-agents-guide) — Developers Digest, 2026.