# How I build software with coding agents

> Source: <https://gist.github.com/ciocan/b6237c646d2ee711f051d63a85bc5bd8>
> Published: 2026-09-23 12:48:43+00:00

A write-up for a friend. This is the setup I have converged on after about a year of running Claude Code and Codex CLI on real products, some of them with the agents working for days at a time while I sleep. It is generic on purpose. Where I quote numbers they come from my own repos, measured on 2026-09-23.

The short version: one Linux box, a terminal multiplexer built for agents, a written contract per repo that tells the agent how work is done, and a loop of grill → issues → orchestrated build → adversarial verification → merge → docs. The agent never commits on its own unless a charter says so, and it never marks something done that it has only read.

**One Linux box at home.** 32 cores, 128 GB RAM, ~900 GB of disk. Everything agent-related
runs here: the agents, the dev servers, the sandboxes, the GPU-fleet control scripts.
Laptops are thin clients.

**Tailscale.** The box has no public address. I reach it over the tailnet from any device.
MagicDNS gives it a name, so `ssh box` works from the laptop and from other machines that
need to talk to services on it (a local Elasticsearch, a local Nominatim, dev servers on
odd ports).

**herdr.** A terminal workspace manager for AI coding agents. It gives me a hierarchy of
workspaces (one per project), tabs (one per concern inside the project) and panes (a shell,
an agent or a server). It detects the agent's status in each pane, so I can glance at a
board and see which agents are waiting for me and which are still working. Both Claude
Code and Codex CLI report their state to it through a session-start hook. There is also a
`herdr` skill so an agent inside a pane can open a tab, spawn another agent in it, run a
command there and wait for a match on its output or for that agent to reach "done". That
is how a coordinator in one pane drives an implementer in another.

**Sandboxes.** Agent products I build use microsandbox VMs for tool execution. Every eval
run leaves VMs and 4 GB sparse disks behind, so cleanup is a written rule, not a habit:
after any eval or dev-server session the agent sweeps orphaned VMs, deletes the directories
no live VM holds, and says in its recap what it cleared. The disk filled up twice before
that rule existed.

**Rented GPUs when needed.** Batch LLM extraction (OCR, structured extraction over millions
of documents) runs on rented L40S boxes with vLLM, driven from the home box by scripts the
agents wrote. The agent launches the fleet, checkpoints progress in the target store, and
audits the result against a gold set afterwards.

**Claude Code** is the main driver. Global config: Fable 5.1 with a 1M context as the
default model, effort `xhigh`, auto permission mode with a written trust-boundary brief
(what is prod, what is a live cluster, which commands are "soft deny" and need a human
even in auto mode).

**Codex CLI** runs beside it, on GPT-6 Astra. Same skills, same hooks, same herdr
integration. I use it for second opinions on plans and for parallel implementation when I
want two independent takes.

**The model split.** This took a few painful iterations to settle:

- The session lead (Fable) plans, dispatches, reads reports, reviews diffs, and merges. It does not write feature code. I have literally stopped it mid-turn with "do not code here, you're the orchestrator".
- Implementers run on Opus, one per issue, each in its own git worktree.
- Adversarial verifiers also run on Opus, one per implementer diff.
- One Fable agent runs the final smoke test of the integration branch after a wave merges.

The reason is the usage window. All-Fable delegation hit the 5-hour cap twice in one evening and killed five running implementers. Opus implementers plus a single Fable smoke at the end gets the same quality for a fraction of the budget. When I name a model in the request, that wins.

**Subagent briefs are self-contained.** A delegated prompt assumes the subagent has no chat
context. It carries the repo path, the objective, the files in and out of scope, the exact
evidence format to return (files, line refs, commands run, diffs, failures, screenshots,
uncertainties), the verification commands, and stop conditions: if the code does not match
the brief, a command fails twice, or the task needs out-of-scope files, stop and report
instead of improvising. Reports are leads, not facts. The lead reopens the cited files
before acting on a finding.

Every repo has a `CLAUDE.md` (or `AGENTS.md` for Codex-first repos). The important design
decision: **it is an index, not a manual.** It says what each surface is, which files own it,
the operative rules, and where the long version lives. A unit test pins its size (64 KB in
one repo, 72 KB in another) and a change may add at most ~10 lines. Narratives, measurements
and incident stories go to `docs/architecture.md` and a `docs/journal/` directory. The old
600 KB version is frozen in an archive file that the agent may grep but never edit.

Per-surface rules load on demand from `docs/agents/*.md` (routes, data, conventions). The
index says "read the matching annex before touching its surface". This keeps the always-on
context small and the detailed rules complete.

The rules that recur across every repo:

- **Issue before code.** The issue is the unit of work. Body: the observed problem without
the proposed fix, acceptance criteria someone else can check, out of scope. Too small
for an issue means too small for its own PR.
- **Branch from `dev`** , named` feat|fix|chore|docs/<issue>-slug` . Never commit to`main` or`dev` . Parallel agents take their own worktree with`.env.local` copied in.
- **The quality gate.** Typecheck, lint at zero errors and zero warnings, format check,
unit tests, the contract check, the build, and the e2e specs that cover the changed
screen. A red gate is never "good enough to review". A proven lint false positive is
suppressed on the line with a written reason, never allowed to flow.
- **Smoke test, never ship a change you have only read.** Start the app, walk the exact
surface through its acceptance criteria and one adjacent flow. Browser: clean console
including React warnings, both themes, both breakpoints, the keyboard path. Record the
exact commands and real output. A test not run is reported as skipped, never passed.
- **Adversarial verification before the PR.** A separate agent with fresh context, briefed
to break the change: "assume it is broken until proven otherwise; for each acceptance
criterion find a concrete input where it fails; report unverified claims, silently
narrowed scope, untested edge cases, regressions in adjacent code". It gets the diff, the
acceptance criteria (not the implementer's summary) and the smoke log, and is asked what
the smoke test failed to cover. Every finding is resolved or rejected in writing.
- **The PR.** Base`dev` , body with the issue reference, what changed and why, how it was
verified with real output, the adversarial findings and how each was settled, before and
after screenshots for anything visual, known gaps. Title is a Conventional Commit because
release-please derives the changelog and version from commit types.
- **No closing keywords.** Never "closes #N" in a title, body or commit. A CI guard matches
the tokens. Issues are closed by a human after acceptance, with a comment mapping each
criterion to the PR that met it. Release PRs once silently closed 13 issues; that is
where this rule came from.
- **Docs in the same change.** README for users, architecture for the deep dive, CLAUDE.md
at ten lines. Any new CLI verb, flag, default or env var updates the operational runbook
in the same PR. A feature is not done until the runbook reflects it.
- **Every long-running CLI shows rich progress.** A bar on stderr with done/total, ETA and
live outcome counters, a one-line startup summary so it never sits silent, plain periodic
lines when stderr is not a TTY, and Ctrl-C prints how to resume. One repo enforces this
with an AST test that refuses hand-built progress objects.
- **Every long job resumes.** Per-unit checkpoints in the target store, cursors instead of
scrolls, a resume that shows as a resume (the bar pre-seeded from the checkpoint), exit
codes 0/1/2/130.
- **Humans commit.** The agent prepares the branch, the commits and the PR body, then asks
before`git commit` ,`git push` or`gh pr create` . Merges are`--merge` , never squash,
so each concern keeps its own Conventional commit. The exception is a written charter for
an unattended run (section 6).
- **Language.** Everything in code is English: identifiers, comments, test titles, commit
messages. The UI language is only for what the user reads. Domain vocabulary that names
a route or a registry concept is the one allowed exception in identifiers.

Skills are the reusable instructions the agent loads on demand. Mine come from four
sources: Matt Pocock's set, the agent-native set, a few from vendors (Elastic, Vercel,
PostHog, Convex) and a few I wrote. A shared `~/.agents/skills` store feeds both Claude and
Codex.

**Planning and scoping**

- `grill-me` . The one I use most. It interviews me about a plan, one branch of the decision
tree at a time, until we agree. Every epic starts with a grilling session; the decisions
it produces go into the epic body as a "Decisions" list.
- `grill-with-docs` . Same, but checked against`CONTEXT.md` and the ADRs, which it updates
as decisions settle. Used on the data pipeline repos where the domain glossary matters.
- `to-prd` and`to-issues` . Turn the grilled plan into a PRD, then into independently
grabbable tracer-bullet issues on GitHub, each with acceptance criteria.
- `plan-arbiter` . When Codex and Claude both produced a plan, compare them, pick one or
merge a hybrid, and output one execution handoff with verification gates.
- `triage` . Moves issues through a state machine and prepares them for an unattended agent.
- `handoff` . Writes the conversation into a handoff document for a fresh agent.

**Execution discipline**

- `efficient-fable` and`efficient-frontier` . The orchestration pattern from section 2:
the expensive model plans, delegates and judges; cheaper subagents do scans, logs, test
runs and bounded edits. Always on.
- `stay-within-limits` . Work in waves of at most three parallel subagents. Between waves
check usage with`ccusage` , pause at 95% of the 5-hour or weekly window, re-check on
resume.
- `plow-ahead` . "Keep going, do not ask." Routine ambiguity becomes a stated assumption;
the recap lists the decisions made. This is what I put in a brief when I go to sleep.
- `agent-watchdog` . Audit another agent's work from a session id, transcript, PR or log.
- `tdd` ,`diagnose` , superpowers'`systematic-debugging` and`verification-before-completion` .
The process skills that stop the agent from patching symptoms.
- `read-the-damn-docs` . Before touching a third-party API, SDK or anything involving auth,
billing or security, search and read the official docs. Never from memory.
- `quick-recap` . Every finished response ends with one status line: green done, yellow a
non-routine follow-up pending, red blocked on me. I read those lines from the herdr board.

**Writing**

- `unslop` . Strips AI tells from every piece of prose (no em dashes, no sycophancy, no
filler, no bold-label bullets, no sentence that would fit another project's docs) and
puts a voice back. A prompt-submit hook re-injects the reminder on every turn so it
survives context compaction. Subagent briefs say "follow the unslop skill".
- `ste-writing` . Simplified Technical English for runbooks.

**Visual and analysis**

- `visual-plan` ,`visual-recap` ,`visualize-repo` . Turn a plan, a PR or a repo into a
local interactive document with diagrams and annotated diffs.
- `graphify` . Any folder or repo into a knowledge graph with community clustering.
- `impeccable` . UI design review and iteration in the live browser.
- The Elastic set: Kibana dashboards, alerting rules, ES|QL log search.

An MCP server is how the agent reads and pokes at the infrastructure without me pasting things. The principle: the agent should be able to see what I can see, in the dashboards I would open myself.

- **playwright** and**chrome-devtools** . Browser automation for smoke tests, screenshots,
console and network inspection, Lighthouse. The Playwright one is what the verifier uses
to walk every signed-in route and capture a screenshot per route.
- **next-devtools** . The Next.js dev server's own view: routes, build errors, docs.
- **shadcn** . The component registry. UI is built from these primitives only, never
hand-written, and the agent looks the component up here before adding it.
- **context7** . Current library docs. Frameworks move faster than model training data.
- **clerk** . Auth SDK snippets.
- **convex** . Read tables, run queries, tail logs, check env vars on the backend. Prod is
read-only for the agent; deploys and mutations stay with me.
- **vercel** . Deployments, runtime logs, env vars, firewall, the sandbox API.
- **posthog** . Product analytics, session replay, feature flags, error tracking, LLM
observability and the alerting. Every metric about my agent product lives here,
including a cost-per-turn alert that mails me.
- **axiom** . Log queries for the data pipelines. (Pipelines log to Axiom; the app product
logs to PostHog. One tool per concern, decided once.)
- **cloudflare** . Workers, R2, observability, for the repos that run edge workers.
- **twenty** . My CRM, over MCP, so an agent can look up or update a customer while working
a support issue.
- **resend** . Transactional mail.
- **workos** and**linear** on the repos that use them.

Plugins from the official marketplace add the language servers (TypeScript, Pyright), the
`code-review`, `feature-dev` and `code-simplifier` agents, the `security-guidance` hooks,
`hookify` for turning a repeated mistake into a blocking hook, and the `remember` journal
(section 7).

Two things I turned off on purpose. The Semgrep plugin needed a headless OAuth that could
not complete and its hook blocked every edit until I disabled it. And I keep the
`context7` plugin off because the user-scope MCP entry already covers it.

1. **Grill.** I open a session on Fable (or Astra) and say "grill me on X". It interviews me
until the design has no open branch. The output is a decisions list with the rejected
alternatives named. On domain-heavy repos this also updates the ADRs.
2. **Spec to tracker.**`to-issues` files an epic and its sub-issues. Each sub-issue has
the problem, acceptance criteria and out-of-scope. Labels tie the sub-issues to the epic.
3. **Orchestrate.** The lead session spawns one Opus implementer per sub-issue in its own
worktree with a self-contained brief. At most three in flight. Implementers never commit;
they leave the tree ready and report with exact commands and output.
4. **Verify.** For each diff, an Opus verifier with fresh context, briefed to refute. It
walks the acceptance criteria in a real browser, signed in as a smoke account I created
(the agent is not allowed to script past the sign-up bot check). Findings get folded
into the same commit or a`fix(scope): … (#n)` commit on the branch.
5. **Gate and smoke.** All gates green, targeted e2e specs named with their counts, a
smoke log with real output.
6. **PR.** One PR per issue in most repos, one PR per epic with a commit per sub-issue in
the ones where I asked for it. I read the diff top to bottom and approve the push.
7. **Final smoke.** After the wave merges, one Fable agent smokes`dev` end to end and
reports green, green with findings, or red. Findings become follow-up issues.
8. **Close and document.** Each issue closed with the criteria-to-PR map. The narrative,
measurements and traps go to the journal, the runbook and, if it changed, the
architecture doc. Post-mortems of production sessions become gists.
9. **Clean up.** Stop dev servers, remove worktrees, sweep sandboxes, say so in the recap.

Numbers from the busiest repo, since 1 July: 2,217 commits, 65% carrying a Claude co-author trailer, commits in every hour of the day, a peak of 153 commits in one day, and the issue and PR counters both past 1,160.

This is the mode people ask about. It is not "let the agent run and hope". It is a written charter plus mechanical backing so the loop survives context loss.

**The charter.** A 27 KB markdown file the top of `CLAUDE.md` points to as the active
unattended charter. It amends a kickoff prompt whose first section is literally
"OVERNIGHT ORCHESTRATOR", with an editable scope line, a 07:00 local wrap-up deadline and
"MORNING DELIVERABLES": a night report, the ledger copied into the journal, a draft PR.
What it pins:

- The orchestrator implements nothing and reads no large files. It dispatches one worker per issue in its own worktree, three in parallel by default, five at most.
- A SMOKE LOCK: one heavy real-data run at a time, ledgered.
- One adversarial verifier per diff, briefed to refute, not approve. Two failed fix cycles
means park the issue with a `backlog` label.
- Tiered verification: a full adversarial pass for pipeline, cluster or CLI changes; one pass for docs and tests. Either tier parks after about two hours.
- Gated self-merge: merge only when the verifier is green and CI is green on the exact head
SHA. It never merges `main` or release PRs. It may close issues only with a three-part
closing comment.
- Stop conditions: the cluster down for more than 30 minutes, disk under 30 GB, a dirty tree at boot, or a step that needs a human. Human-only steps (live writes, scheduling, prod deploys) are queued as paste-ready commands, never executed.
- Operating under the charter is the commit, push and merge authorisation that the global rule otherwise withholds.

**Files are memory.** The charter's anti-amnesia contract says the conversation, including
any summary of earlier context, is not to be trusted. The working set is three files:
`ledger.md`, `queue.md`, `state.md`. Ledger before the next task. A re-anchor ritual at
session start, after every compaction, and every ~90 minutes: re-read the charter and the
three files, append `re-anchored <time>, resuming <task>` to the ledger. Commit the ledger
at least every three hours so a crash costs at most three hours of record.

**Hooks back it up.** `SessionStart` and `PostCompact` hooks cat a JSON block that says, in
effect, "context was just compacted, re-read the charter and the ledger before the next tool
call". The agent cannot forget the ritual because the harness re-injects it.

**What it produced.** On the data pipeline repo: 14 dated run days with ledgers totalling
5,300 lines and 95 re-anchor rows, times taken from `gh`, `git` and the log store rather
than self-reported. Four night reports. A 34-day streak of consecutive commit days. 116
commits on the peak day and 397 in the peak week. Half of all commits carry an agent
trailer, and in the peak month it was 363 of 632.

**Steering while it runs.** I steer through three things: amendments to the charter text,
an operator queue of paste-ready commands I run or refuse in the morning, and a progress
artifact the agent refreshes (a percent-complete page with the issue list). Interactive
sessions on the same repo read the orchestrator's ledger and message it before starting
anything cluster-heavy, because two agents on one shared machine once clobbered each
other's workspace. That incident became a pinned workspace id in the repo settings and a
paragraph in the contract.

**Mission briefs for bounded runs.** For narrower unattended jobs I write a kickoff file
instead of a charter: "You are a fresh session. Do not commit. Test each prompt before
moving to the next. Maintain this log file. Stop instead of patching." One such brief drove
17 sequential payment-flow test prompts. Another is a parser convergence loop: one bounded
change per iteration, stop and escalate after 30 iterations without progress, finish with a
handoff section in the design doc. A hackathon entry ran on an overnight mandate: at 23:28 I
told the coordinator to push the score, spin up more GPU boxes and fine-tune a 9B model,
keep the packaged submission valid at all times, and commit as it went. 50 of its 56
commits carry the agent trailer, running from 13:00 through the night to 11:00.

The other kind of unattended time is not coding. It is the agent driving a fleet: 10 to 20
rented GPU boxes running vLLM over millions of documents, or a multi-hour crawl, or a
re-index. The rules that make this safe are the CLI rules from section 3: resumable with
checkpoints in the target store, progress on stderr with resume hints, dry-run by default
with `--apply`, adaptive supervisors in tmux panes over halves of a shard plan. After the
run the agent audits a sample against a gold set that was built by two frontier models and
validated by hand, and one repo has a "blind rebuild" check where a second agent rebuilds
the analysis from scratch to compare outputs.

Three layers, because each one fails differently.

**The agent's own memory.** Claude Code keeps a per-project memory directory: one file per
fact with a type (user, feedback, project, reference) and an index file loaded every
session. Feedback entries carry the why and how to apply, so a correction I gave once
("Opus agents wrote non-English identifiers, rewrite the trailer with filter-branch")
survives across sessions. Codex has its own memory store.

**The remember plugin.** A journal per repo in `.remember/`. Hooks on every tool use
extract the session, a small model summarises it into `now.md`, an hourly pass rolls it
into `today-<date>.md`, a daily pass into `recent.md` (seven days) and `archive.md`. At
session start the recent files are injected, so a fresh session knows what the last one
was doing without me explaining. After a compaction only the identity file comes back,
which is exactly why the charter's re-anchor ritual exists.

**GitHub and the repo.** Everything that matters to a human lands in one of: the issue
(problem, criteria, closing comment with the criteria-to-PR map), the PR body (reasoning,
verification with real output, adversarial findings and how each was settled), the journal
(`docs/journal/<issue>-<slug>.md`, 246 files in one repo), the runbook (every CLI verb,
every env var, every recovery), the architecture doc (measurements), and gists for
post-mortems of production sessions and for research write-ups. The rule for CLAUDE.md
itself is ten lines per change, so the index never becomes the manual again.

- Write the contract before the first feature. The size cap and the "index, not manual" rule are the two decisions that kept it useful for a year.
- Adversarial verification is not optional. The verifier gets the criteria and the diff, never the implementer's summary, and its first question is "what did the smoke test not cover".
- Humans commit, in interactive mode. The charter is the one place where that changes, and it changes with named gates.
- Files are memory. Anything the agent must not forget lives on disk and is re-read after compaction by a hook, not by good intentions.
- Cheaper models implement, the expensive one judges. Usage windows are real.
- Every long CLI resumes and shows progress. You will Ctrl-C it, and the agent will too.
- Cleanup is a rule with a recap line, or the disk fills up.
- Ship one wave a day with the evals green. I gate agent-product PRs on a corpus replay of delivered answers so a new refusal rule cannot silently break old ones.
- Keep the eval harness in the repo. My agent product has a question bank with hard budgets, an LLM judge, a nightly core run in CI and a cost alert in PostHog. That is what makes "the agent got better" a measurable claim instead of a feeling.
