Loop Engineering — a deep technical explanation with realistic examples

Loop engineering is a deterministic control loop around an AI coding agent — schedule, durable state, and verification — that lets developers move up from prompting to designing the machine. Boris Cherny, Head of Claude Code at Anthropic, said: "I don't prompt Claude anymore. I have loops running that prompt Claude and figure out what to do. My job is to write loops." The key principle is that a loop is only as trustworthy as the parts of its verification the agent cannot fake.

The idea is often credited to Boris Cherny Head of Claude Code at Anthropic , who put it bluntly: "I don't prompt Claude anymore. I have loops running that prompt Claude and figure out what to do. My job is to write loops." This document goes past the slogan into the mechanisms: the control-theory framing, the exact algorithms that make a loop safe, the concurrency internals, the cost math, and the epistemics of why an agent cannot be trusted to check itself. Loop engineering is building a deterministic control loop around an AI coding agent — schedule + durable state + verification — so the agent drives the moment-to-moment cycle and you move up to designing the machine. The single load-bearing principle: A loop is only as trustworthy as the parts of its verification the agent cannot fake. "Am I stuck?" and "is this correct?" must be decided by deterministic code or a genuinely independent checker —neverthe agent's own self-report. Everything below is a consequence of that sentence. A control system regulates a process toward a setpoint using feedback: setpoint goal │ ▼ ┌──────────────┐ command ┌──────────────┐ │ controller │ ──────────► │ process │ │ your loop │ │ agent + repo │ └──────────────┘ └──────────────┘ ▲ │ │ measurement │ └─────────── sensor ◄─────┘ verifier / tests / metrics Setpoint = the goal "open PRs stay green", "no failing CI on main" . Process = the agent acting on the repo. Sensor = the verifier / test suite / numeric check — how you measure the gap between reality and setpoint. Controller = the loop logic that turns the measured gap into the next command. The critical distinction from control theory: | Open-loop | Closed-loop | | |---|---|---| | Definition | Act, never measure the result | Act, measure, correct | | Agent analogy | "Prompt → hope" | "Prompt → verify → decide → repeat" | | Failure mode | Drifts arbitrarily far from goal | Bounded by sensor quality | A raw agent call is open-loop. You prompt, it acts, and nothing measures whether the result matched intent — you do, manually. Loop engineering closes the loop by making the sensor verification and the controller decide/escalate part of the automated system. The corollary that governs everything: a closed loop is only as good as its sensor. A control loop with a broken thermometer will happily drive the room to 1000°C while reporting "72°F, all good." That broken thermometer is exactly what an LLM asked to grade its own work is see §4 . Three nested scopes. The jump from harness → loop is defined by adding schedule + state + verification . Context engineering → what goes in ONE prompt "one move" Harness engineering → ONE agent's environment: tools, "one game" permissions, rules — a single session Loop engineering → harness + SCHEDULE + STATE + "the tournament" VERIFICATION, running over TIME - Miss the schedule → it's a one-off script open-loop, runs once . - Miss the state → the agent is amnesiac every run and re-tries failed things forever no integral term — no memory of accumulated error . - Miss the verification → the sensor is missing; the loop is "closed" only in appearance. Three properties of language models make the naive "just let it run" approach fail. The loop machinery exists to counter each one. - Statelessness across sessions. A model has no memory between separate runs. Whatever isn't written to durable storage is gone. → Requires external state §5A . - Uncalibrated self-assessment. A model's confidence is not a reliable estimate of its correctness. The same reasoning that produced a bug produces the sincere belief that the bug is correct. Confidence and correctness are drawn from correlated distributions, not independent ones. → Requires an independent sensor §4 . - Autoregressive momentum / no natural stop. Given "keep trying until CI is green," a model will keep generating attempts — including plausible-looking non-fixes — indefinitely, because "produce the next token" has no built-in notion of "this is hopeless, stop." → Requires an external circuit breaker §5B . Loop engineering is, precisely, the set of external mechanisms that supply the memory, the sensor, and the stop condition that the model lacks. This is the load-bearing wall. Everything else is plumbing. Let M be the maker produces a solution and V the verifier judges it . Verification adds information only if V's errors are independent of M's errors. If V = M same model, same context, same prompt lineage , then any mistake M is blind to, V is equally blind to. The verification is a rubber stamp — it multiplies confidence without adding evidence. This is why "run the tests once and mark it done" is not verification if the same session wrote the tests and the code: a weak test written to pass weak code confirms nothing. Loops lean on a hopeful asymmetry: checking a solution is often cheaper and more reliable than producing one it's easy to verify a returned sort is ordered; harder to write the sort . When this holds, an independent checker is a cheap, strong sensor. But the asymmetry inverts when the artifact under review is itself the distortion. The canonical case: Someone ran a hedge fund as a fully autonomous loop with an LLM verifier judging backtests. It failed because a strategy's failure mode isoverfitting— the strategy looks brilliant on historical data precisely because it was accidentally shaped to fit that exact history. A second LLM reading the backtestcannotcatch the curve-fit, because the backtest it's reading isalreadythe overfit artifact. The lie and the evidence-of-the-lie are the same document. The fix was to replace the LLM verifier with a numerical, non-overridable check that measures something the maker structurally cannot fabricate: | Test | Sharpe | Verdict | |---|---|---| | In-sample training data | +2.54 | looks like a winner | | Out-of-sample held out | −4.33 | collapses | | Numerical checker | — | REJECT — no trade | The +2.54 is exactly the number that would make a human — or an LLM verifier — ship it. The out-of-sample split is data the maker never touched , so it cannot have been fit to. That untouchable measurement is the entire product. The maker/checker split is worthless unless the checker measures something the maker cannot fake. In code, that's an independent re-derivation of correctness a real test run, a type check, a compile, an EXPLAIN ; in statistics, it's out-of-sample, cost-aware, multiple-testing-penalized numbers — never a second model's vibe. | Layer | Mechanism | What it makes unfakeable | |---|---|---| Am I stuck? | Deterministic circuit breaker §5B | The agent can't claim "progress" — the numbers decide | Is it correct? | Separate verifier sub-agent, default REJECT | The maker can't approve its own diff | Is it real? | Non-LLM checks: tests run, EXPLAIN, compile, OOS split | The maker can't fabricate the measurement | read STATE ─► TRIAGE ─► IMPLEMENT maker sub-agent, isolated worktree situation? worth │ ▲ doing? ▼ │ VERIFY checker sub-agent / numeric check │ │ persist STATE ◄─ gate/escalate ◄─┘ write down safe? or human? State is a STATE.md or DB row / ticket board answering exactly three questions — and deliberately nothing more, to avoid rot: - What are we working on now? - What did we try last time, and what was the outcome? - What is waiting on a human? In control terms this is the integral term : it accumulates the history of error so the controller can act on trends , not just the latest instant. Without it, the loop is memoryless and will re-attempt the same failed fix on every run. State rot is the dominant failure here: the file references merged PRs, closed tickets, or dead branches, and the loop acts on ghosts. Mitigations: a prune step every run, a Last run timestamp, validating IDs against the live API, and one state file per loop pattern no shared unstructured file . This is the most technically interesting component. It answers "is this agent stuck?" with pure code — no LLM in the decision path , so it's cheap to run every iteration and impossible for the agent to argue past. Data model: interface Attempt { iteration: number; action: string; // short description of what was tried outcome: 'success' | 'failure' | 'noop'; error?: string; // raw error / stack trace if failed tokensUsed?: number; } interface Ledger { goal: string; attempts: Attempt ; } // oldest → newest Step 1 — normalize errors to a stable signature. Raw errors differ every run even when they're "the same" failure: "Error at db.ts:42 connection refused 0x7ff8a2 " ← attempt 1 "Error at db.ts:88 connection refused 0x3aa10b " ← attempt 2 errorSignature strips the volatile parts so the same failure maps to the same key: ISO timestamps → <ts 0x… hex addrs → <addr absolute paths → basename only :line:col → removed any other digits → collapse spaces Both examples above collapse to Error at db.ts connection refused — identical signatures. This is the crux: without normalization the loop would think every retry is a "new" error and never notice it's stuck. Step 2 — measure similarity with character trigrams Jaccard . For actions and near-miss errors, exact-match isn't enough, so it uses set overlap of overlapping 3-character substrings: Jaccard A, B = |trigrams A ∩ trigrams B | / |trigrams A ∪ trigrams B | → 0.0 disjoint … 1.0 identical default threshold ≈ 0.85 counts as "the same" Trigrams are robust to trivial rewording "cannot connect" vs "can not connect" in a way exact string compare is not, and cheap enough to run every iteration. Step 3 — check triggers, cheapest & most-actionable first. It walks the trailing run of failures consecutive failures since the last non-failure and returns the first condition that holds, so the reported reason is the most useful one: | Order | Trigger | Fires when | Why this order | |---|---|---|---| | 1 | stagnation | same error signature ≥ 3× in a row | most specific, most fixable | | 2 | frustration | near-identical action repeated ≥ 3× in a row | semantic looping | | 3 | no-progress | ≥ 5 consecutive failures, no success between | broad "going nowhere" | | 4 | token-budget | cumulative tokens ≥ cap | cost ceiling | | 5 | max-iterations | ≥ 10 iterations | absolute backstop | The ordering is deliberate: stagnation "you keep hitting the same wall" gives a human the most actionable escalation message; the iteration cap is a dumb backstop of last resort. Step 4 — prune + inject. Before the next iteration it builds a compact context block to prepend to the prompt so the agent doesn't repeat itself: Loop Context managed Goal: <goal Progress: iteration 7 · 0 ok · 7 failed · 340k tokens Already tried do NOT repeat : - bump pg driver to 8.11 - add retry wrapper around connect Failure patterns: - 5× Error at db.ts connection refused Most recent error pruned : ← stack trace truncated to N lines ... Circuit breaker: OK 7/10 iterations used . Pruning keeps only the last N attempts, truncates stack traces to a few lines, and collapses repeated identical failures into one entry with a repeated count — so a 40-line log of the same error becomes one line, keeping the next prompt cheap and focused. Separate from the breaker. The breaker knows run history ; the gate knows static policy — and the two are deliberately kept apart single responsibility; history logic lives in exactly one place . Config gate.yaml : version: 1 denylist: globs that must never be touched without a human - " /secrets/ " - "infra/prod/ " - " / key " maxFiles: 20 more than this in one change → escalate autoMergeAllowlist: for auto-merge, EVERY path must match one of these - "docs/ " - " / .test.ts" checkGate action, paths runs three checks, most-severe first: denylist — any changed path matches a forbidden glob → BLOCK. file-count — more paths than maxFiles → BLOCK a huge diff is a red flag regardless of content . allowlist auto-merge only — any path not on the allowlist → BLOCK. Subtle but important: glob matching uses dot: true . Without it, won't match a path segment starting with . , so /secrets/ would silently miss .secrets/prod.json and / key would miss .aws key — exactly the hidden files a denylist most needs to catch. This is the kind of one-character bug that turns a safety control into security theater. When multiple agents run in parallel, two editing the same file is merge hell. Each agent gets its own git worktree shares history, separate working tree . On top of that sits an advisory file-lock system so two agents don't grab overlapping paths: - Each owner writes .loop/locks/<owner .json = { paths, lockedAt, expiresAt? } . Cross-process atomicity for the check-then-write critical section uses an exclusive-create mutex file: open path, 'wx' fails if the file exists and is atomic on POSIX and Windows. Without this, two lock calls racing could both pass the "is it free?" check before either writes — the exact collision the feature exists to prevent. Path overlap is a deliberately-simple segment-by-segment glob compare src/ covers src/a/b.ts ; a wildcard segment matches anything . Advisory, not a full glob engine. TTL / expiry : locks can auto-expire so a crashed agent doesn't wedge the system forever; a sweep reclaims expired locks. Deadlock detection : waiting agents write .wait.json entries recording who they're blocked on. Before waiting, the system runs DFS cycle detection over the wait-for graph and throws Deadlock detected: A → B → A instead of hanging forever. Classic OS deadlock avoidance, applied to agents. When the breaker trips or the gate blocks, the loop must stop and notify — not silently retry. The failure mode here is "escalation only writes to a state file no one reads." Real escalation pings a human through a connector Slack, a Linear comment and parks the item in a Waiting on human section, ideally with an alert if it sits there 24h. runs per day = 86,400,000 ms/day ÷ interval ms tokens per day = runs per day × tokens per run blast radius per day ∝ runs per day × P bad action × mean damage So going from a 1-day to a 5-minute cadence is 288× the runs — and 288× the cost and 288× the blast radius. A one-off agent making a bad edit is annoying; a loop making it 288×/day unattended and merging it is an incident. Not every run costs the same. A realistic model weights three outcome types: cost per run = P noop × tokens noop nothing to do — cheapest + P report × tokens report found something, described it + P action × tokens action ran maker + verifier — most expensive The mix shifts with autonomy level more autonomy → more actions → higher cost : L1 ≈ {noop .6, report .4, action 0 } L2 ≈ {noop .5, report .3, action .2} L3 ≈ {noop .4, report .35,action .25} Triage cheaply first. Spawn expensive maker+verifier sub-agents onlywhen state says there's something actionable. Empty watchlist → exit in <5k tokens. A loop that runs the full sub-agent chain on every empty tick is the 1 way to turn a 5-minute cadence into a runaway bill. "Why We Killed Our CI Sweeper After Day 4": ran every 5 min on a broken branch → 8M tokens in 48h , proposed 11 fixes, 3 were fake symptom-fixes, 1 broke prod config . Root causes: no report-only phase, verifier in the same session as the maker, no budget cap, no branch allowlist. Every one of those is a missing mechanism from §5. Autonomy is earned incrementally, each rung adding a mechanism and proving it before the next: L1 REPORT ONLY looks + triages, takes NO action. Run 1–2 weeks. Measure: is the triage even accurate? ↓ add: separate verifier, worktree, attempt cap, human-approved merge L2 ASSISTED FIXES proposes changes; human approves merges. ↓ add: path denylist, token budget, metrics, human gates on risk L3 UNATTENDED acts alone within hard, machine-enforced limits. A readiness scorer can encode this as a gate, not a suggestion : it scans the project for ~35 signals state file? verifier skill? gate.yaml ? budget doc? run log? evidence of real runs in git history? , sums weighted points to a 0–100 score, and maps score → level roughly L1 ≥ 38, L2 ≥ 58, L3 ≥ 78 . Crucially, an L3 score is capped back to L2 unless it also finds: a verifier, a budget file, a run log, AND git evidence you actually ran the loop. You cannot score your way to "unattended" without proof you earned it. | | Primitive | Supplies | Counters from §3 | |---|---|---|---| | 1 | Scheduling | the heartbeat cron, /loop 5m , CI on a timer | — makes it a loop | | 2 | Worktrees | parallel isolation + locks §5D | orchestration collisions | | 3 | Skills | written-down intent: conventions, build cmds | intent debt / statelessness | | 4 | Connectors MCP | reach real tools: GitHub, Jira, Slack, DBs | — makes it useful | | 5 | Sub-agents maker/checker | independent verification §4 | uncalibrated self-assessment | | 6 | Memory / State | the durable spine §5A | statelessness | Build the minimum viable loop first : schedule + one triage skill + a state file, report-only. Add each further primitive only after the previous version proved its value and revealed its failure modes. Worktrees when you start making changes; the verifier when you start acting autonomously; write-scoped connectors last. Goal: keep open PRs green and reviewed without a human polling GitHub. cadence: 10–15m during work hours state: pr-babysitter-state.md row per PR: last SHA, attempts, status loop turn: 1. TRIAGE cheap : gh pr list --json number,headRefOid,statusCheckRollup → PRs whose checks are RED and whose SHA changed since last run 2. if none actionable → exit in <5k tokens §6.3 early exit 3. for each actionable PR: acquire worktree lock on that PR's paths §5D no collision MAKER sub-agent: smallest diff to fix the failing check CHECKER sub-agent stronger model, "find reasons to REJECT" : actually RUN npm ci && npm test and report output §4 real sensor BREAKER: 3rd attempt on same failing check → STOP, escalate §5B/5E GATE: checkGate 'commit', changedPaths before pushing §5C 4. persist STATE + append loop-run-log.md Guardrails doing real work: attempt cap kills infinite-fix loops; verifier runs tests kills "verifier theater" ; worktree lock kills parallel collisions; gate kills over-reach into denylisted paths. Watches a text-to-SQL service and self-heals bad generated SQL — with a checker the LLM cannot fake directly applies §4.2 . trigger: event = "generated SQL failed / returned suspicious result" MAKER: LLM regenerates SQL from the question + schema + prior errors CHECKER: NON-LLM, non-overridable: - EXPLAIN parses without error syntax/plan valid - runs on a READ-ONLY replica with LIMIT bounded, safe - row count / column types sane vs the question's expected shape - planner cost estimate < threshold no full scan on a 2B-row fact BREAKER: same SQL-error signature 3× → escalate to a human analyst §5B STATE: questions that repeatedly fail → surfaces schema / prompt gaps Why it fits the root: an LLM asked "does this SQL look right?" is verifier theater. EXPLAIN + a bounded run on a real replica measures something the generator cannot fabricate — the same lesson as the quant backtest, in SQL. cadence: 6h–1d scope: PATCH + low-risk CVE only for the first 30 days blast-radius control MAKER: bump one dependency → one PR CHECKER: full npm ci && npm test IN A WORKTREE real, not "looks fine" GATE: majors + denylisted packages → human, never auto §5C allowlist cadence: 1d, weekdays, via CI on a timer skill: triage-only — structured output, ONE line per item + suggested action action: NONE. Writes findings to STATE.md; human reviews weekly. cost: ~50k tokens/day one light run purpose: the report-only rung — prove the triage is accurate before ANY autonomy | Failure | Symptom | Root cause | Prevented by | |---|---|---|---| | Infinite fix loop | same PR "fixed" 5+ times, never converges | no attempt cap; weak verifier | circuit breaker §5B + separate verifier | | Verifier theater | "looks good" but CI red | same model/context; vague prompt | independent checker that runs tests §4 | | State rot | acts on merged PRs / dead branches | no prune; unread state | prune step + Last run + ID validation §5A | | Token burn | bill spikes on empty ticks | full chain on every run | cheap triage + early exit + budget §6.3 | | Over-reach | edits unrelated / forbidden paths | permissive skill; no denylist | gate.yaml denylist + smallest-diff §5C | | Parallel collision | two agents edit same file | no isolation | worktree + advisory lock §5D | | Escalation failure | stuck loop, human never told | max-attempts unimplemented; silent state | breaker + connector ping §5E | | Comprehension debt | ships faster than anyone can read | human stopped reading | mandatory review + weekly digest | | Cognitive surrender | "the loop handles it," no opinions | success metric = volume | human gates + metric = quality-held | Intent debt — the agent's cold-start ignorance of your conventions. Fix: skills, written once, read every run. Comprehension debt — the loop ships code faster than you can read it, so you stop understanding your own repo. Fix: mandatory human review of non-trivial diffs; a weekly loop digest. Cognitive surrender — you stop having opinions on correctness. The knife's edge: designing loops with judgment is the cure; using loops to avoid thinking is the accelerant — same action, opposite outcome. Orchestration tax — the human cost of coordinating N parallel agents. Worktrees remove the mechanical collisions, but you remain the ceiling on how many loops you can actually absorb and review. - The thing that decides "done/correct" is not the thing that produced the work, and measures something the producer can't fabricate. - There is a hard, machine-enforced cap on iterations, tokens, and repeated failures — the loop escalates instead of spinning. - Every run reads and writes durable state ; nothing important lives only in a model's transient context. - High-risk paths are on a denylist enforced in code , not in a prompt's good intentions. - Parallel workers are isolated and cannot silently clobber each other. - Autonomy is earned up a ladder , each rung proven before the next. - A human is pinged on escalation through a channel they actually read. The harness ships every primitive natively, so you rarely need bespoke CLIs — you need the discipline : | Loop primitive | Claude Code equivalent | |---|---| | Scheduling | cron / scheduled tasks / /loop | | Worktrees | isolation: "worktree" on sub-agents | | Skills | Skills / SKILL.md | | Connectors | MCP servers | | Maker/checker | sub-agents Agent tool , a stronger verifier | | Circuit breaker / budget | Workflow token budget + loop-until conditions | | Memory / state | file memory / a committed STATE.md | Bottom line: treat the agent as an unreliable actuator inside a control loop you own, where every trust boundary is decided by deterministic code or a genuinely independent checker — build it like someone who intends to stay the engineer, not just the person who presses go.