{"slug": "i-handed-41-tasks-to-an-ai-loop-the-bottleneck-was-judgment-not-code", "title": "I Handed 41 Tasks to an AI Loop. The Bottleneck Was Judgment, Not Code", "summary": "A developer reported that an AI loop processing 41 backlog tasks across two repositories found the bottleneck was judgment, not code. The developer split task processing into judge, build, and human roles, with the judge re-evaluating tasks before dispatch, closing 6 tasks through decisions and withdrawals and correcting state errors. The approach used git worktrees and fast-forward merges instead of pull requests, with the human providing final approval.", "body_md": "Is there a region at the bottom of your task ledger you haven't scrolled to in weeks?\n\nMine held 41 tasks across two repositories. I use AI agents every day, and yet the ledger never shrank.\n\nOne morning I ran my homegrown \"list the tasks that are ready to start\" command. The answer was empty.\n\n```\npython3 ~/.claude/scripts/claims.py ready\n# (no output)\n```\n\nForty-one tasks, and zero of them ready to hand to an implementation session. Implementation capacity was sitting idle. The bottleneck was judgment.\n\nThis article is a field report from one week of processing those 41 tasks with an AI loop split into three roles: judge, build, and human. By the end, you should be able to tell where your own ledger is actually stuck — and what to design first if you want to run an unattended loop.\n\nThis is a record of two repositories, one week, one person (n=1). Every number comes from logs and commits; generalize only within that range.\n\n## Tasks start rotting the moment you write them\n\nThe day before I built the loop, I hand-dispatched 7 tasks to implementation sessions as a trial.\n\nTwo of the 7 had premises that had already collapsed by the time work started. A spec they depended on had changed, or the problem itself had been dissolved by some other change.\n\nA ledger's \"someday\" entries assume the world as it was at write time. A task that sat for a few weeks needs a re-judgment — \"is this still worth doing?\" — before anyone implements it.\n\nIn other words, processing a ledger has a judgment layer that comes before implementation. If your automation design skips it, the AI will stack correct code on top of rotten premises.\n\nI did two things. First I split task processing into three roles; then I laid the plumbing to run them unattended.\n\nThe role split:\n\n| Role | Who | Concretely |\n|---|---|---|\n| Judge | A resident session on a stronger model (one tier above the build side). One per repository | Re-judges every task in the ledger and dispatches only the ones still alive |\n| Build | A fresh session spawned per task | Implements exactly one task on a git worktree. Never touches main |\n| Human | Me | Final approval on merge, drop, and filing only. The last switch |\n\nNo pull requests. Build sessions just stack commits on a task branch.\n\nAcceptance is done by the judge session — `git diff --stat`\n\nplus re-running the tests — and once the human approves, it fast-forward merges. The list of unmerged branches doubles as the acceptance queue.\n\n```\ngit branch --no-merged main   # unmerged = acceptance queue (the PR substitute)\ngit merge --ff-only task/<name>   # only after the human says \"merge\"\n```\n\nFor a one-person operation, PR review UIs and merge buttons were overkill. Branches plus ff-only merges are enough to build the structure where a human checks last.\n\nHere is the balance sheet from the first full day with all three roles running (separate from the 7-task trial the day before; operation at this point was still manual — unattended mode comes later).\n\nFor the arithmetic-minded: 2 of the 27 closed tasks belonged to an adjacent repository, fixed in passing, so they sit outside the 41. That gives 41 − 25 + 6 = 22, and the books balance.\n\nWhat stands out in the 27 is the share closed without writing code.\n\nThe judgment pass alone — before anything was dispatched to implementation — closed 6 tasks through decisions and withdrawals. Read-only investigation sessions closed 4 more.\n\nThe judgment pass also repaired the ledger in ways other than closing. Three tasks sat marked blocked even though their dependency had completed weeks earlier; one had a resume condition that structurally could never fire.\n\nAll of them moved forward with nothing but a state correction.\n\nA ledger shrinks through judgment before it shrinks through implementation. If your tasks look piled up, the first suspect may not be a shortage of implementation capacity — it may be that this re-judgment is nobody's job.\n\nTo be clear, the build side worked too: 14 implementation sessions (pilot included) ran in one day under a parallelism cap of 3, producing the 13 merges.\n\nWhat was missing was not implementation muscle but the layer that judges tasks into a dispatchable shape. That is how to read these numbers.\n\nThe briefing document I hand to each build session (I call it a kickoff packet) failed me twice.\n\n**First: every packet I wrote on day one was wrong somewhere.** A premise that was only half true, or a prescribed fix that wouldn't actually close the hole.\n\nThe countermeasure was to put \"Phase 0: re-verify the premises. If falsified, stop and report instead of implementing\" at the top of every packet. I stopped writing packets as orders and started writing them as hypotheses.\n\nThe same sessions then began producing correct results from incorrect instructions. And having them report the falsifications they found feeds material back to the judge.\n\n**Second: when I enumerated the review steps in a packet, an omission was read as permission to skip.** One build session did skip the simplification review — the one step I hadn't listed.\n\nThe fix: stop enumerating steps, reference the conventions instead, and state explicitly that \"anything not written in this packet defaults to the conventions.\"\n\nThe specifics you write into a delegation document get read as exemptions for the ones you didn't. At least this build session interpreted the list that way.\n\nEverything so far was manual operation. Next came unattended mode — and my first design failed.\n\nClaude Code can schedule recurring runs inside a session, and that is what I reached for first. But an in-session timer shares its fate with the session.\n\nA restart, a tool update, the session's own lifespan — any of them stops it, and nothing on the outside can detect that it stopped.\n\nA loop that fails loudly is manageable. A loop nobody notices has stopped is the dangerous one. So I settled on this design principle:\n\n**Timers live outside the session. Judgment lives inside. And the human's answers arrive only inside the session too.**\n\nConcretely, macOS launchd (the OS-native cron equivalent) fires a small shell script every few days.\n\nThis script (I call it the tick) never reads the ledger. Its entire job:\n\n```\n# tick's job: find a live judge session; if none, spawn one; then request one cycle\n# (the real thing is ~/.claude/scripts/triage-tick.sh; excerpted to the essentials)\nif ! find_live_triage_session; then\n  spawn_session && rename_to_fixed_name\nfi\n# digest = the report the judge session sends the human at the end of a cycle\n# (the Slack notification in the next section)\nsend_prompt \"Unattended triage cycle. Go as far as the digest.\nDo not merge, publish, or touch rules / hooks / security gates.\nFile nothing and drop nothing on your own.\"\n```\n\nThe intelligence stays in the session; the tick only checks for a pulse and wakes it up. Keeping the plumbing dumb is a deliberate choice — fewer parts that can break. (Durability itself is what the upcoming scheduled runs will test.)\n\nAn unattended cycle sends two kinds of Slack messages: one notification per item that needs a human decision, and a single liveness line that is always sent at the end of the cycle.\n\n.claude triage cycle done: 1 decision pending (…)\n\nIf the scheduled time passes and that line doesn't arrive, something in the loop is dead.\n\nConverting silence into an anomaly signal — that turned out to be the real substance of going unattended.\n\nSlack, however, is one-way here. Treating replies as human answers would open impersonation and misreading paths, so answers like \"merge\" are accepted only at the judge session's own screen.\n\nIn fact, on day one the judge session nearly misread a suggestion Claude Code had auto-inserted into its input box as a human instruction. Deciding which single channel carries the human's words is worth settling before you go unattended.\n\nThe first cycle produced a discovery I hadn't planned for.\n\nMy environment has machine gates that run on every commit (secret scanning and the like). The gate keeps a human-approved list of scripts allowed to run unattended — and that list had gone stale 2.5 weeks earlier. **The gate had been quietly dormant.**\n\nNobody had noticed.\n\nA gate that fails red at least gets noticed. A quietly dormant gate keeps waving things through while pretending to be green.\n\nOnly after building the unattended loop did I see that \"is the loop alive\" and \"are the verification devices the loop relies on alive\" are the same problem.\n\nAddy Osmani, in [Loop Engineering](https://addyo.substack.com/p/loop-engineering) (June 2026), points at the leniency of letting the model that wrote the code grade itself, and at human verification throughput as the ceiling on parallelism. This week traced both points on the ground.\n\nI would add one thing: the verification devices themselves belonged on the watch list. Osmani's \"done is a claim, not a proof\" applies to gates too.\n\nHonestly: this system has only been verified up to the entrance.\n\nAs of this writing, open tasks stand at 22 across the same two repositories (2 ready, 1 awaiting a decision, 19 waiting on conditions). That is the result of 25 closed and 6 newly filed over the week.\n\nAnd the first unattended cycle delivered exactly one decision request to the human.\n\nBefore automating, my job was to stare at tasks, remember them, start one, and get interrupted by another. Now my job is to answer the one message that arrives.\n\nImplementation can be delegated. Most judgment can be delegated too.\n\nWhat remains at the end are short words: \"may I merge this?\" and \"can we stop doing this one?\"\n\nYour 41 tasks are probably not waiting on implementation either, for the most part. Open just one — the oldest blocked task — and check whether its dependency is still alive.\n\nBuild the loop later if you like; the size of the discovery won't change.\n\nThis system depends on my local environment (macOS, launchd, homegrown session tooling), so copying the steps won't transplant it. If you want to try it, paste the following into your coding agent as-is, and have it do read-only investigation and planning first.\n\n```\nDraft a plan to apply this article's design to my environment. Do not implement yet.\n\nPreliminary investigation (read-only):\n1. Survey my current task tracking (files, tools, task count, last updated)\n2. Identify the OS-native mechanism available for unattended scheduled runs\n   (launchd / systemd timer / Task Scheduler, etc.)\n3. Check whether a notification channel (Slack or similar) is available for sending\n\nThe plan must include:\n- A structure that separates judgment (task re-triage) and implementation\n  into different sessions/processes\n- Wiring that places the timer outside any interactive session\n- A one-line liveness signal at the end of each cycle, and a way to detect its absence\n- The boundary of what the unattended side must NOT do\n  (no merging, no publishing, no config changes)\n\nPresent the plan and get my approval before proceeding to implementation.\n```\n\n", "url": "https://wpnews.pro/news/i-handed-41-tasks-to-an-ai-loop-the-bottleneck-was-judgment-not-code", "canonical_source": "https://dev.to/shimo4228/i-handed-41-tasks-to-an-ai-loop-the-bottleneck-was-judgment-not-code-23dp", "published_at": "2026-08-20 10:23:59+00:00", "updated_at": "2026-08-20 10:45:15.285684+00:00", "lang": "en", "topics": ["artificial-intelligence", "ai-agents", "developer-tools"], "entities": ["Claude"], "alternates": {"html": "https://wpnews.pro/news/i-handed-41-tasks-to-an-ai-loop-the-bottleneck-was-judgment-not-code", "markdown": "https://wpnews.pro/news/i-handed-41-tasks-to-an-ai-loop-the-bottleneck-was-judgment-not-code.md", "text": "https://wpnews.pro/news/i-handed-41-tasks-to-an-ai-loop-the-bottleneck-was-judgment-not-code.txt", "jsonld": "https://wpnews.pro/news/i-handed-41-tasks-to-an-ai-loop-the-bottleneck-was-judgment-not-code.jsonld"}}