I separate two things that often share a name. A software factory automates product delivery. Someone has decided the work is worth doing, and the factory ships it. An AI development team also automates discovery, and decides why and what to build. This article is about the factory.
Most descriptions of software factories keep design with people, who write the spec and cut it into small tasks for agents. I think a factory should start from the issue as its requester wrote it. The factory picks the issues to work on, settles their requirements, designs, plans, implements, reviews, opens the pull request, waits for CI and merges. A person sets the product goal, answers the few decisions only a person can make, and judges each sprint's result. Nobody reviews the pull requests before the factory merges them.
That works when every check a person usually makes is replaced by a mechanism, and when each mechanism fixes boundaries and evidence while leaving the path between them to the agents. The rest of this article walks through the loop, the mechanism behind each check and why it is needed, and how to build and grow a factory like this.
Work starts from a product goal, a short text the person keeps on a dashboard and rewrites when priorities change. The lead agent reads it with every open issue in the tracker and proposes a sprint: one purpose, the issues that purpose needs, which of them build on which, and the key results that will show the purpose reached. The person approves the sprint on a card.
Each issue then runs on its own branch, in its own worktree, and issues that don't depend on each other run in parallel. The lead settles what the issue requires and writes a design, and a reviewer checks it. The lead splits the work into tasks, and an executor implements them one at a time. The reviewer runs a code review and a security review, and the executor decides what to do with each finding. The factory then pushes the branch, opens the pull request, and waits for CI, which includes the end-to-end suite, and for any review bots, whose findings a reviewer judges. A gate checks that everything holds for the exact commit and merges it.
When every issue has merged, been held for the person, or been dropped, the sprint ends. An analyzer collects what happened. The lead judges each key result, reviews the team's working agreements, and proposes the next sprint. The person approves all of it together, and the next sprint starts once the changed agreements have merged.
Anything that fails along the way, a step that can't finish or a review that won't converge, goes to the lead as an exception, and the lead decides how to move it on. The person hears about it only when the decision is theirs.
The loop closes without a person because each check a person normally makes has a replacement:
| What a person usually does | What the factory does instead |
|---|---|
| Decides what to work on next | The lead plans each sprint from the product goal and every open issue |
| Clarifies what an issue really asks for | The lead settles the outcome, requirements and non-goals from the issue, the code and the documents, and asks only about behavior nothing settles |
| Checks the design | A reviewer checks it against the requester's words and the code, independently of the lead |
| Reviews the code | Separate code and security reviews, with each finding judged before it's acted on |
| Confirms the behavior works | CI runs the end-to-end suite as a required check |
| Decides when to merge | A gate merges the exact commit that passed, and holds the few categories a person must see |
| Steps in when something goes wrong | The lead resolves exceptions at the level of their cause |
| Improves how the team works | A retrospective reviews the team's own agreements every sprint |
Most instructions in a factory fall into one of two kinds. A boundary narrows where an agent may operate, such as keeping a public contract, taking no irreversible action without authority, implementing the requested outcome while keeping the non-goals out, or finishing only when the result is observable. A work-generating rule creates obligations, such as producing three alternatives, writing unit, integration and end-to-end tests for every feature, resolving every review finding, or stopping when a field is missing.
Both look like safeguards. A capable model follows the second kind faithfully and produces exactly the artifacts, tests, checks and stops it implies, whether or not the outcome needs them. In a factory nobody is watching to say that's enough, so a work-generating rule runs at full volume on every issue.
So the factory's rules open with the boundaries and the evidence each step owes, and every mechanism below follows them:
## How to judge your work
- Be strict about boundaries and evidence, and flexible about the path between them.
The boundaries are: the issue's outcome and non-goals, existing public contracts
and acceptance criteria, what the human approved, the authority this file gives
you, and the evidence your step must leave.
- A design is the current means, not a boundary. It is written before the code that
tests it, so parts of it can be wrong: whoever finds a part wrong on evidence
corrects it on the issue's branch, the code and the design document together,
while the boundaries hold. A design merges only with the implementation that
makes it true.
- Name the outcome before choosing the means. "No change" and "reuse what exists"
are valid conclusions when the evidence shows the outcome already holds.
- Treat findings, suggestions, and discovered possibilities as candidates. Keep only
work the outcome, a boundary, a real consumer, or necessary proof requires.
- When a result falls short, doubt what it rests on before adding to it: find the
premise or mechanism that is wrong and change it at the level it lives.
The person judges the factory's work a sprint at a time. A sprint has one purpose because one purpose is something a person can judge at the end. Reporting per issue would put the person back into every loop. The lead takes only the open issues that purpose needs, which keeps it from filling a sprint with easy issues that move nothing, and sets each key result at the line below which the purpose isn't reached. Key results set at the best result imaginable turn every miss into another run.
The person owns the outcome, and the agents own how the codebase is built. AGENTS.md says the second part outright: the agent owns this codebase and will keep building on it, so what it leaves behind is its to maintain. The line between the two is written down, because without it every agent escalates by its own sense of caution:
## When to involve the human
The human decides only:
- product behavior no source decides, when choosing it changes what users get.
The sources are the issue, the acceptance criteria, the merged design documents,
the rule the code already applies, and how the codebase settled the same cause
elsewhere. A defect fix that makes the code do what its rule intends decides
nothing new. The design step writes an open choice as the issue's question before
designing, and the factory asks;
- a change to the sprint goal or an issue's stated outcome, or a departure from what
the human approved: a prototype, or the external connection, infrastructure, CI,
or contract change of a design they approved;
- what only the human can do: sign in, grant access, spend money, or supply
information no record holds.
Everything else, a design correction included, is resolved from the code, the
documents, and the tests.
The list of sources does as much work as the list of decisions. In one trial sprint, a design followed the rule the code already applied to the same cause elsewhere, without naming that precedent. The design review saw product behavior with no source, and the lead turned it into a well-argued question for me. I sent it back with one line: "This isn't a human decision. Follow the precedent and decide it yourself." Precedent went onto the list of sources after that question, and the sprint's retrospective added a working agreement that the design names the precedent it relies on.
Decisions reach the person as cards on the dashboard, and only the work that depends on a card waits:
| Card | Requested when |
|---|---|
| sprint | a sprint is proposed |
| spec | a design reaches product behavior no source decides |
| design | a reviewed design adds an external connection, changes infrastructure or CI, changes the schema, or breaks a contract consumers outside the repository rely on |
| prototype | an issue adds a new screen, built with mock data before implementation |
| merge | a pull request touches a category held for the person |
| review | the sprint is over: result, retrospective, next sprint |
A card approves an action. Approving a design means proceeding with that external connection, and the document stays open to correction afterwards.
Issues arrive bloated, vague, or aimed at the wrong outcome. A capable model reconciles all three into a coherent plan and builds it faithfully, delivering exactly what was asked for when what was asked for was wrong. So before designing, the lead settles the issue: one observable outcome, the non-goals, and each requirement with its source, marked as either what exists today or what the issue asks for. A mechanism the issue suggests stays a candidate until the outcome needs it. When the code shows the outcome already holds, the lead closes the issue with the evidence.
A design is a prediction made before the code that tests it, and parts of it will be wrong. If a design is approved and fixed before implementation, a wrong prediction becomes the standard everything else is checked against, and the work built on it fails or collects workarounds. So each issue has one branch and one pull request, carrying its design document and its implementation together, and whoever finds part of the design wrong on evidence corrects the code and the document together on that branch.
A reviewer checks the design before any code is written. It reads the issue as its requester wrote it, separately from the lead's reading, and treats a requirement the lead added or a scope it dropped without a reason as a finding. It then checks, in order, that the design delivers the outcome, agrees with the documents that govern the same contract, rests on claims that hold in the code, states enough to build and test, and adds nothing whose removal would still meet the outcome. On one issue in a trial, it found that none of the planned tests exercised a path the design promised, so every test could have passed with that path unchanged.
The lead then splits the work into tasks, one executor launch each. Every task ends in an executable check, a test or command with the result that proves the task, because an executor without one has no way to know it's finished and nobody to ask. The first task proves the riskiest boundary the design names, so a wrong premise surfaces while little is built on it. The executor owns how each task gets built. When the code shows part of the design wrong, it makes the smallest correction that keeps the boundaries and fixes the design document in the same commit.
Nobody reviews the factory's pull requests before they merge, so CI is the last executable check, and the end-to-end suite is what shows behavior that only appears across boundaries, such as a page, the API behind it and the database working together. Before a factory starts, the repository needs an end-to-end suite running in CI as a required check on the main branch. When a level of the product has no test that can fail, nothing in the loop shows a break at that level, and the factory merges it.
Once the factory runs, the opposite pressure arrives. LLMs add tests freely. The agent implementing a task writes tests for it, the agent fixing a finding adds a test for the fix, and every layer proves the same behavior again. Nobody stops it in a factory, the end-to-end suite grows with every issue, and CI becomes the bottleneck of the whole flow.
The control is ownership. Each behavioral claim has one owning test, at the narrowest boundary where the claim is observable, and a wider test proves only what its boundary adds. The skill that governs integration and end-to-end tests states when a case earns its place:
Select an integration or E2E case only when all of these hold:
1. an acceptance criterion, preserved behavior, or the task's done condition names an
observable claim;
2. the claim depends on a module, persistence, process, browser, or service
boundary; the cases of a calculation, parsing, or branching rule inside it stay
with that rule's unit test;
3. no existing test owns the claim at that boundary with assertions that detect
its material failure;
4. the failure it detects harms the people who use the product and would stay
unseen without this boundary, at a cost that harm justifies.
An empty selection is a valid result and needs no record.
The design and the design review select the cases, and the implementation carries out what was selected. A cap such as three browser tests per feature looks like a simpler control. It binds the agents with a number no requirement backs, as a second brake on what the selection conditions already decide.
For a claim about appearance, the end-to-end test saves a screenshot at the state the acceptance criterion describes. The reviewer opens it and judges it against the criterion's wording and the design, and a screenshot that contradicts the criterion is a finding. It needs no baseline image.
Which failures deserve a test comes from a ranking of harm in AGENTS.md, which every agent loads on every task. Its shape:
## What This Product Must Keep True
Weigh every change, test, and review by the harm a failure does to the people who
use it, in this order:
1. Exposure: nobody sees what they must not, nobody acts as someone else, and
nothing that authenticates appears in logs or responses.
2. Stored data: what people saved is not lost, or overwritten by anyone but its
owner.
3. What others rely on: links, APIs and integrations already in use keep working.
4. Daily use and appearance.
The order weighs harm when something breaks; what to build next is decided elsewhere.
Tests, reviews and the judgment of review-bot findings all weigh failures by this order. A test, a check or a mechanism earns its place by value to users or by maintainability over the medium and long term.
A reviewer can always find something more, and a writer told to address every finding will. Each round is technically defensible, and the loop never asks whether the result got better for anyone. Making the reviewer authoritative and the author obedient turns review into a ratchet.
The code review and the security review run as separate launches, so a security concern isn't diluted by everything else a code review looks at. The reviewer works in its own session with its own prompt, treats another step's account of its work as a claim to check, and raises a finding only for a confirmed break, with its location, its basis, its effect and the smallest correction. The executor answers each finding in one of three ways, with a skill based on reviewee-judgment:
### Decide
Judge every finding with reviewee-judgment. Group findings that share a cause and
respond once to the cause.
- Apply a confirmed break of the outcome, an acceptance criterion, the design's
contracts, a repository rule, or correctness. Take the smallest response that
removes the cause: remove or narrow first, then reuse, then add.
- Apply a security finding when leaving it would break an explicit requirement or
repository rule, or leave a concrete failure reachable in the real trust model.
- Decline a finding that adds scope, reverses a non-goal, duplicates proof that
exists, asks for optional hardening, or costs more than its effect. Give evidence
the reviewer can check: the file and line, the test, the requirement.
- Return it to the lead when the response would change the outcome, a non-goal, or
what the human approved.
On the next round the reviewer gives each earlier finding a status (resolved, withdrawn, or maintained) and raises a declined finding again only with evidence that the decline's reason doesn't hold. After two rounds the lead decides, and the pull request body records any finding left standing with the lead's reason.
AI review bots get the same treatment, more strictly, because they raise whatever is technically possible without weighing its cost. Once CI passes, the factory waits for them. Findings a bot itself rates optional are declined outright, and a reviewer judges the rest:
- apply only when the finding shows a failure in the order AGENTS.md gives that this
change's current code produces, and you can name the input or state that triggers it.
- decline the rest: appearance, optional hardening, style, scope beyond the issue,
proof that already exists, or a problem the code no longer has.
- return only when resolving it would change the issue's outcome, a non-goal, or
what the human approved.
The factory posts a decline's reason as the reply after "Not applied:", so it states
the fact that settles the finding (the file, the test, or the requirement) and
nothing else.
A bot that leaves no trace within a few minutes of the wait starting counts as not installed, so the flow works the same with or without one.
A wrong merge can't be taken back, so the gate is deterministic. It runs after the required CI checks pass on the exact commit the factory pushed. The lead declares in its design which approvals an issue needs, and the script also classifies the changed paths, so an undeclared change still gets caught. The gate merges only when every card the change needs is approved, and it holds the categories a person must see, such as a schema change, for the person's merge card.
The person answers held merges at the sprint review, and holding costs little when sprints run in hours. A merge card doesn't ask the person to read the diff. It has a button that starts the issue's worktree environment, and the person checks the behavior there.
Right before merging, the script checks that the branch is still current with the base and that its head is the commit whose checks passed, then merges that commit. If either has moved, the issue goes back to push and CI. A merge approval holds for the branch as the person saw it, and the factory asks again if an agent changes the branch afterwards.
Classify by path only where a path is a reliable signal. A new screen is hard to detect from paths in most codebases, so the lead's declaration and the design review cover it. A rule that depends on what a change does to a file has to judge the whole change to that file. One of mine matched only added lines in a deployment file, and a review bot pointed out that a change deleting the line that kept production private would have passed it.
A step that fails, is blocked, or runs out of review or CI rounds goes to the lead as an exception. The lead looks for the cause the way a person stepping in would:
### Find the cause
1. Did the work fail, or the process? A process that died, timed out, or hit a
transient error with the work otherwise on track needs only a retry.
2. Trace from the symptom. Compare it with a state that works: the same check on
origin/<base>, a comparable feature, the last green commit.
3. Place it at its level, highest first: the sprint goal or an issue's outcome; the
design; the plan; the code; the environment. A step that failed twice the same
way rests on something above the code; doubt the plan or the design before
retrying it again.
It then takes one way forward at that level, from a retry with the cause written down, through replanning or redesigning, to dropping the issue, or asking the person when the way out is theirs.
The four roles map to the kinds of work in the loop:
| Role | Does |
|---|---|
| lead | Plans sprints, settles requirements and writes designs, builds prototypes, plans tasks, resolves exceptions, writes the sprint review, talks to the person |
| executor | Implements tasks with their tests, applies review findings, merges the base branch into the issue branch when it conflicts |
| reviewer | Reviews designs, code and security in separate launches; judges review-bot findings; checks the retrospective |
| analyzer | Collects the facts a retrospective needs, without judging them |
The lead is the only agent that talks to the person and writes to the tracker, so the person deals with one voice. There is no coordinating agent. Routing work from one step to the next is the same decision every time, and the script makes it.
Agents never talk to each other. Each step is an item, a markdown file on a board, and one item is one launch:
---
step: <the step this item runs>
to: <role>
status: open | doing | done | blocked | failed
issue: <tracker key, when the step serves one issue>
worktree: <the worktree the step works in>
refs: <paths or items the step acts on>
out: <a file the step writes for the factory to read>
verdict: <pass | findings, set by a reviewer>
---
<what to carry out, written by the factory>
## Result
<written by the agent>
The script sets open and doing. The agent finishes by setting done or blocked, and a reviewer its verdict, through a small CLI that accepts only a few fields and values. The prose result is for whoever reads the item next. What the script acts on goes through these fields or through JSON files at known paths, so the script never interprets prose.
The script doesn't take done at its word either. A review can't finish without a verdict, a step that leaves uncommitted changes becomes an exception, and each step's output is validated before the next item is filed. Those checks are structural. Whether the design and the code are right is the reviewer's judgment.
An overloaded or rate-limited model is retried a few times. Any other failure, such as a usage limit or an expired sign-in, would recur on every launch, so the factory s itself and posts the reason. A cap on launches per hour does the same when planning or reviewing keeps repeating, which usually means something is stuck. After a restart, steps that were mid-run simply run again: their commits are still in the worktree, an implementation step first checks whether its done condition already holds, and the factory remembers the pull request it opened.
Model choice is a cost decision. If cost matters, give the strong models to design and review, the lead's design ahead of the reviewer. The executor can run on a cheaper model, and the price is time: implementation runs slower, so throughput per hour drops. If cost doesn't matter, run every role on the same strong model. If you worry that a design and its implementation share the same blind spots, give only the reviewer a different model. My factory runs the lead on Claude Opus 5.5 at high effort, the executor on GPT-6 Luna at max, the reviewer on GPT-6.1 Sol at high, and the analyzer on GPT-6.1 Terra at high.
Parallel issues mean parallel servers, databases and test runs on one machine. Each worktree gets its own port and local data, and a way to run one command against its own app that starts the app for that command if it isn't running, because the agents' sandbox ends the processes a command leaves behind.
Give the agents the permissions the company or team already allows an agent to have, and run the factory inside that scope. Nothing in this loop needs more. Neither CLI runs with a flag that skips permissions. Hooks stop direct attempts to merge, push, record an approval or touch production, and they apply only to agents the factory launched, so people's own sessions work as before. A hook catches honest mistakes. An agent set on merging could call the API through a script, and extending the pattern list for each new spelling is the patch problem again, so the guard that holds against production is keeping production credentials away from wherever the factory runs.
Approvals have the same limit. The dashboard's approval API runs on the same machine, so any local process can reach it. Separating them properly, so an agent couldn't approve even on purpose, would mean running the agents as a different operating-system user from the dashboard, and for an internal factory I judged that not worth its cost. What remains is a record. Each approval is kept as a file with a timestamp, which doesn't prevent a forged approval but lets the person spot one they don't remember giving.
A common view holds that agent work should be made deterministic, with each correction turned into a check and the whole flow scripted up front. I agree with half of it. A deterministic rule decides in advance what the model may do, and early on nobody knows which parts of the path are stable. Hardening early fixes the use case. Everything locked has to be maintained, tested and worked around, often to protect things that never needed protecting, and every case the rule didn't foresee becomes a stop or a question for a person. The cost is autonomy, which is what keeps people out of the loop.
So let agents make the calls first, run the flow on real work, and read the logs. The factory appends one line when each launch starts and one when it ends, and keeps each agent's session:
{"at":"2026-11-16T10:29:51+09:00","role":"lead","item":"20261116-102951-design-OPS-318","step":"design","sprint":"20261116-0756-export-history","issue":"OPS-318","log":"...","cli":"claude","model":"claude-opus-5-5"}
{"at":"2026-11-16T10:41:07+09:00","role":"lead","item":"20261116-102951-design-OPS-318","ended":true,"code":0,"log":"..."}
Count the launches by role and step, then read what the frequent ones actually changed:
jq -r 'select(.ended | not) | "\(.role)\t\(.step)"' .factory/launches.jsonl | sort | uniq -c | sort -rn
Move a step into the script when its output is fully determined by its input, and leave the others with agents.
The AI team I ran before the factory showed both directions. Its coordinating agent first watched the board itself and forgot to wait again after each pass, so waiting moved into the script. A day of its launches, counted this way, showed more than twenty that only received a finished task and filed the next one from a plan that already named who did each step. Filing moved into the script along with a few jobs like it, its prompt got shorter, and the coordinator's cost per hour fell from $0.67 to $0.34. Then the script went one step too far. It counted how often a goal's plan was revised and ended the goal at the second revision. Counting was deterministic, but ending a goal was a judgment, and I took that rule back out.
Three things are deterministic from the start, because their failures can't be undone: the merge gate, the hook that stops an agent from recording an approval, and access to production.
Most repositories a factory moves into already have an AGENTS.md, skills, a review process, CI and hooks, and people who keep working there. Judge the existing skills by the bar the factory needs, since nobody reviews its pull requests. One that falls short can stay for the people who use it while the factory reads its own. Shrink AGENTS.md to principles, because it loads on every task for people and agents alike.
Keep the team's gates: CI checks, review bots and command-blocking hooks run on the factory's pull requests too, and one comes off for the factory only once there's evidence the factory holds its quality without it. Only people change the factory's rules, AGENTS.md and the agent configuration, and the gate holds any pull request that touches them.
The order to build it in:
A factory gets things wrong, in its first sprint and in later ones. When people build something for AI, they often expect it to be right the first time and read each failure as proof that agents can't do the work. A factory's accuracy comes from reviewing each sprint's failures and keeping what was learned, so the same failure doesn't happen again. What gets kept has to be chosen, and only the lessons that will matter in later sprints stay.
When the factory does something wrong, look for what in place caused it before adding anything. A contradiction between two instructions, an instruction that is wrong, or one that invites the behavior explains most failures. "Resolve every finding" produces the review loop it was meant to prevent. In an early trial run, a reviewer's step said to finish a bot review without a verdict, while the factory's CLI refused any reviewer item without one, and the run stopped. The lead traced the stop to the factory and raised it to me, because only people change the factory's rules. Checking every instruction against the code then turned up more problems that would have stopped a run.
When something must be added, place it by whether it can be checked. A rule that can be checked mechanically and won't change goes into a deterministic check. A rule that needs judgment goes into a skill, or into AGENTS.md if every task needs it. Everything else the agents work from is placed by who needs it and when:
| Kind | Place | Read |
|---|---|---|
| What the product must keep true, who owns what, governing sources | AGENTS.md | every task, by people's sessions and the factory alike |
| How to implement, test, document and respond to review | skills | when a step names them |
| Roles, authority, approvals, the board | the factory's rules and role files | every factory launch |
| What a step must do and produce | the step's file | that step |
| What the team learned about its own work | working agreements | every launch of that role |
| Facts about one area of the code | a knowledge folder | before changing that area |
| Anything decided the same way every time | the script | never, by agents |
Whatever gets added, write it as the state you want. A rule written against the failure that triggered it ("don't do X when Y") covers that one case. The next failure is a slightly different case, another patch follows, and the conditions keep growing. Every condition costs context in every launch and adds a branch the model reasons through on every decision. A rule that states the result you want covers many cases nobody has seen yet, because the model works out what the result implies.
Deterministic checks are the easiest to write as patches. A lint rule against a pattern that always breaks states a result that happens to be checkable. A check that blocks the exact command that went wrong last time is a patch, and the next spelling of the same mistake gets past it.
At the end of each sprint an analyzer collects the facts without judging them: the plan against what happened, each issue's review rounds and failures, the person's words, and, for each working agreement, whether its problem occurred and whether the choices it governs applied it. The lead judges each key result from what merged and its tests show, hands anything only a person can judge to the person with the evidence, and reviews the agreements:
Review every working agreement by what its problem did this sprint:
- The problem occurred: find why the agreement did not prevent it. When it was not
applied where the choice is made, move it there. Otherwise change what produces
the problem at its level, then keep, rewrite, or drop the agreement.
- The problem did not occur and its occasion arose: keep it when the choices there
applied it; drop it when they did not.
- Its occasion did not arise: drop it when it names a particular event, or when the
problem it answers would cost little if it occurred.
Add an agreement only for a gap the sprint showed that none answers, at its cause
rather than the particular event. Every agreement is another branch every launch
must keep.
In one trial sprint the retrospective kept a single agreement, the one about naming precedent. Its draft had two more, which were dropped because they repeated what the rules and skills already said. Another sprint's retrospective added nothing.
When the retrospective changes an agreement, a reviewer checks it, and the person approves the sprint review, the agreement changes and the next sprint together. The next sprint starts only after the agreements merge, because agents read them from the base branch.