llm-assembly-line

A developer has published a pattern for running long, raw, dictated work orders through AI coding agents, called 'llm-assembly-line'. The method involves cutting a brief into work packages, each passing through a build station and two independent inspection stations, with a rework loop that only ends on passed inspection. The pattern builds on Andrej Karpathy's LLM Wiki pattern for persistent context and is designed to let agents work autonomously on entire batches of feedback and change requests.

A pattern for running long, raw, dictated work orders through AI coding agents. One brief goes in. It gets cut into work packages. Each package passes a build station and two independent inspection stations. Nothing leaves the line uninspected. BRIEF ──► cut ──► build ──► inspect ──┐ ▲ │ └────── rework ◄─────┘ │ ──► REPORT Most people hand an agent one wish at a time. That works, and it wastes the thing agents are actually good at. The alternative is to hand over a whole batch at once: everything you noticed this week, dictated raw, one point after another, three hundred lines if that is what it takes. Do that and the brief stops being the hard part. The hard part becomes the instruction for how the batch gets worked. That instruction is what follows: cut into work packages, one builder per package, independent inspectors that try to break the work, a rework loop that only ever ends on a passed inspection, decisions made on the agent's own authority instead of questions back to you, and a final report you can check line by line without having to remember what you ordered in the first place. I use this on every sizeable brief and it has held up. Take it, change it, make it yours. The line itself is my working method. Its foundation is not. For an agent to work a three-hundred-line brief reliably, it needs context that is not reassembled from scratch in every session. The structure that provides it is Andrej Karpathy's LLM Wiki pattern : https://gist.github.com/karpathy/442a6bf555914893e9891c11519de94f https://gist.github.com/karpathy/442a6bf555914893e9891c11519de94f That idea is his, not mine. In short: instead of re-deriving knowledge from raw documents on every question RAG , the agent builds and maintains a persistent, interlinked markdown wiki in three layers — immutable raw sources, the wiki the agent writes, and a schema the two of you evolve together. Knowledge gets compiled once and then kept current. Read his gist first if you want to understand the foundation. What you have here is the floor built on top of it. An agent harness with subagents. Claude Code, Codex, Pi or equivalent. Without separate independent contexts the inspection collapses, because the same agent reviews its own work and predictably finds it good. Recommended: a maintained wiki structure in the project , per the pattern above. The line runs without one, but then the agent works from whatever it can scrape together in the moment instead of from what the project knows about itself. Step 7 of the line is the other half of that deal: it feeds what each brief taught back into that structure. - Dictate or write your entire brief in one message. Feedback, change requests and new features, one point after another, raw and long. Give too much context rather than too little, and do not polish it. - Append the prompt below to that message. Or drop it into the project once as a file say llm-assembly-line.md and end your brief with: Work this order per llm-assembly-line.md . - Everything from The order above is a long, raw, dictated brief onward is addressed to the agent, not to you. One warning about step 5 of the line: it decides ambiguities on its own instead of asking you. That is the point of the thing, and it means you have to actually read the final report. The decisions are listed there, and that is where you overturn them. Everything inside the block is the prompt. Copy the whole block, without the explanation around it. The order above is a long, raw, dictated brief: feedback, change requests and new features, listed one after another but written in one pass. Work through it completely and on your own authority, following this line. Two properties separate this from ordinary work: 1. It runs start to finish without stopping. Questions back to the user are the exception, not the route. 2. Nothing counts as done until an independent check passes. Not even a fix. 1. Cut the brief into work packages Read the whole brief before you cut the first package. Every self-contained point — one piece of feedback, one change request, one feature — becomes one work package with a running number WP1, WP2, … , the wording or a close paraphrase from the brief, and the files it touches. Nothing is silently merged or dropped. A point that turns out to be moot stays a package and is explained as such in the report. Write the package list into a ledger file ASSEMBLY-LINE-<date-or-round .md , wherever the project keeps its documents . The ledger, not the chat, is where status, inspection findings and decisions live, package by package. 2. Cut waves by file conflict, not by importance Packages that touch the same files never run at the same time. Cut the packages into waves so that inside one wave no two packages share a file. Concurrent agents in one file overwrite each other's intermediate state. 3. Staff the line Set up a workflow the harness's workflow tool if it has one; otherwise individual subagents; otherwise sequential work in the main context — in that order . Per work package: 1. One builder agent implements the package. It gets the wording of that one point, the files it touches and the project's conventions — not the whole brief. 2. One independent inspector for code quality checks the result adversarially: correctness, tests run the existing ones, add the missing ones , edge cases, conventions. Independent means its own context, no access to the builder's reasoning, and a mandate to refute rather than to confirm. 3. One independent inspector for UI and UX checks every package that touches an interface: usability, states empty, loading, error , labels, consistency with what is already there. A package with no interface surface records that in the ledger instead of skipping the inspection silently. 4. The rework loop always ends on a passed inspection If an inspection finds a defect, a new build run starts for that package, and the inspection then runs again . The loop never ends on a fix: the last step of every package is always a passed inspection. A change nobody has checked is not a finished package, it is an open one. If a defect is handed over to a different package, it counts as fixed only once the inspection passes there . After three unsuccessful runs, stop turning the package. Record the state, the findings and the remaining gap in the ledger, and mark the package as partially done in the final report. 5. Blockers: decide, do not ask Whatever used to trigger a question — an ambiguity in the dictation, two possible routes, a missing detail — you decide yourself. Pick the option that comes closest to the visible intent of the brief and is easiest to reverse, and record the decision, the reason and the rejected alternative in the ledger . The user reads the decisions in the final report and can overturn them there. Three things stay exempt and justify a stop: destructive, hard-to-reverse actions deleting, force-pushing, changing permissions ; leaving the project you work in foreign directories, foreign repositories, foreign systems ; and bypassing a permission prompt. The last one is never allowed, not even to take a measurement. 6. Whole-system check When all packages are through, check how they fit together. Run the project's full test suite and type check. Look for interactions between packages, because two packages can pass alone and contradict each other together. Then read the brief once more from the top against the package list: is every point really assigned to a package, and is every package either finished or honestly marked open? 7. Document, ingest, synthesize If the project keeps an LLM wiki or a comparable maintained knowledge base, its schema applies: update the pages that changed, file durable insight as a synthesis, update the index and the log, put open questions on the agenda. If the project keeps none, the yield belongs in whatever documentation structure it has. The brief itself is a raw source: if there is a place for original documents, put its wording there before it sinks into the chat history. 8. Final report — easy to digest, still complete The report follows Simplified Technical English ASD-STE100 , applied to the language of the brief: short sentences, one statement per sentence, active voice, one term per thing, no filler. The user must not have to remember anything, so the report lists every package, including the ones that went smoothly: | Per package | Content | |---|---| | Order | What was asked for, in one sentence, close to the wording of the brief | | Result | What was built and where it lives files, pages | | Inspection | What the inspectors found and how it ended | | Your check | How the user verifies the result, in one or two steps | After the table, collected: the decisions you made on your own with their reasons, the open or partially done points, and the state of the check chain tests, type check, build with numbers. Hard rules of this line 1. No package is reported done without a passed final inspection. 2. No two concurrent agents in the same file. 3. No permission prompt is bypassed. Whatever comes out of that does not count. 4. Every independent decision is recorded. None is left out. 5. The final report names all packages, not only the notable ones. Use it, change it, rebuild it. :