Agents by function, not by role Developer Vicmaster released banneret, an open-source multi-agent kit that organizes agents by function rather than by job title, after four trials on real projects. The design pairs a frozen spec, read-only parallel scouts, a single-threaded builder, a verifier fan-out returning CONFIRMED/REFUTED/PLAUSIBLE-UNVERIFIED verdicts, and a mandatory integration pass, drawing on Cognition's Devin experience, the Berkeley MAST study of seven frameworks across 200+ tasks, and Anthropic's multi-agent research system, which the author notes costs roughly 15× the tokens. In the trials, banneret found three ship-stoppers in a security feature and falsified its own riskiest fix in CI, and a cold-reader agent given only framesmith's 1,278-line README completed twelve tasks in about eighty seconds while surfacing documentation examples that fail the parser they document. Agents by function, not by role TL;DR: The multi-agent systems that imitate a dev team PM agent, developer agent, QA agent don’t survive contact with real work, and the research says why: failures come from under-specification and missing verification, not from model capability. So I built the opposite shape. Agents by function : a frozen spec, read-only scouts, exactly one builder, a verifier fan-out, and a mandatory integration pass. Then I put it through four trials on real projects before believing it. It found three ship-stoppers in a security feature, falsified its own riskiest fix in CI, and ran on a second tool, where its first run both worked and taught the kit something. The kit is now open source: banneret https://github.com/vicmaster/banneret . Last Thursday I asked a simple question: what sub-agent system could develop any kind of project? The honest answer required reading before building, and the reading is sobering. Cognition’s experience with Devin https://cognition.com/blog/multi-agents-working is that parallel writers diverge on implicit decisions and produce broken integrations. Their follow-up this year concedes that a narrower pattern works: many agents contributing intelligence while writes stay single-threaded . The Berkeley MAST study https://arxiv.org/abs/2503.13657 analyzed seven multi-agent frameworks across 200+ tasks and found failures concentrate in specification gaps, inter-agent misalignment, and missing verification. That’s system design, not model limits. And Anthropic’s multi-agent research system https://www.anthropic.com/engineering/multi-agent-research-system shows fan-out winning big for research , while warning that it’s weak for tightly interdependent work like code, and costs ~15× the tokens. Put those together and the shape is obvious. It isn’t a team of job titles. It’s a pipeline of functions: 1. Spec. Contracts frozen before anyone writes code I use spec-kit https://github.com/github/spec-kit . Changing the spec mid-build means stopping and re-freezing, on the record. 2. Scouts. Cheap, read-only, parallel. They return conclusions with file:line references, never file dumps. And only for foreign context: upstream docs, ecosystem, repos you won’t touch. 3. One builder: your main session. Full context, single-threaded. Sub-agents never write. 4. Verifiers. The fan-out that actually pays. Each one gets a falsifiable brief specific claims it should try to prove wrong and returns CONFIRMED / REFUTED / PLAUSIBLE-UNVERIFIED verdicts, each with the command it ran or the lines it read. It also returns a “checked and not flagged” section, so you can calibrate it. A second, narrow round runs on the fix diff, because the riskiest code in any task is in the fixes. 5. Integrate. Full suite, spec/docs reconciliation, one end-to-end read of the diff. Never skipped. Plus a budget per phase with a hard stop line. My first bounded multi-agent trial earlier this year ran 4× over budget silently , and silent is the part that hurts. A diagram is a hypothesis. Here’s what four real runs did to it. Trial 1: a verifier that tests docs by acting on them The pipeline’s most unusual agent is the cold-reader . It receives only a document: no code, no web, no project context. Its training is the one floor you can’t remove. Then it has to complete falsifiable tasks using nothing but that document. Where it fails is where the doc fails. I pointed it at framesmith https://github.com/vicmaster/framesmith ’s 1,278-line README: one agent, twelve tasks, about eighty seconds. It answered all twelve correctly. The interesting part was its “I genuinely can’t tell what this means” audit, which surfaced things no style linter would: - The README’s own multi-line examples, pasted literally, fail the parser they document “Unmatched parentheses” . The doc and the code contradict each other, and you can run the contradiction. - A “five presets” claim where four exist. - A tool that re-keys every ID on bind, without the tool’s own section ever saying so. Just as important: it did not flag the dense-but-precise passages. A reviewer that only finds problems can’t be trusted. One that also tells you what’s fine can be calibrated. Everything went back to the repo as two issues 227 https://github.com/vicmaster/framesmith/issues/227 , 228 https://github.com/vicmaster/framesmith/issues/228 for that repo’s own agent to fix. More on why that matters below. Trial 2: a real security feature, end to end Next, the full pipeline on a real task in reeve https://github.com/vicmaster/reeve : complete key-rotation support for its tamper-evident audit ledger. Key epochs, chained epoch records, epoch-aware verification, a rotation procedure. The verifier fan-out confirmed nine findings before merge . By the builder’s own admission, three of them would have shipped from a straight single-session build. There was no control run, so read that as the builder indicting itself, which is the convincing direction. The three: - A permanent false BROKEN verdict after two back-to-back rotations. Epoch records are immutable, so permanent really means permanent. - The epochs model reading the default database instead of the ledger’s. - A digest test that computed its expected value with the function under test. A green test proving nothing. Two verifiers found the same partial-keyring bug independently. And the cold-reader, pointed at the upgrade notes, caught a genuine operator trap: the old ledger key derives from secret key base , so you must capture it before rotating that secret. That’s the kind of thing the author can’t see, because the author already knows it. The budget data was humbling in a useful way. Fan-out phases ran ~3× my estimates: the whole task was budgeted at 810k tokens and took ~1.39M. But the total landed at 1.7×, monitored and reported , against the 4× silent overrun of my earlier trial. The playbook now just says it: estimate fan-out at 3× your first intuition. It shipped as reeve 0.7.0 https://rubygems.org/gems/reeve . Trial 3: verification you can’t argue with One finding stayed PLAUSIBLE-UNVERIFIED : a MySQL isolation race, fixed by reasoning alone because I had no local MySQL. What the agent did next was the best move any verifier made all week. It built a throwaway branch with the fix disabled and pushed it through CI. The MySQL job failed on exactly the three new specs, each in the predicted way. Then it showed the real PR green on the same specs, and deleted the branch. Red without the fix, green with it. That’s not “the tests pass”; that’s falsification. It’s now a rule in the verifier contract: a fix you can’t verify locally doesn’t get a shrug, it gets a plan to prove it in both directions. Trial 4: same kit, different tool “Portable” is a claim until it runs somewhere else. So I compiled the same contracts for Codex https://developers.openai.com/codex/subagents , which has native sub-agents as TOML files. There, my scout’s access: read becomes sandbox mode = "read-only" . That’s enforcement by configuration instead of by prose, at least on paper the honest caveat is below . Then I ran the pipeline on a real issue in my CRM: directory linking across every entry path. At its center was a genuine contract decision: match by email, then by company+name, and on ambiguity, never link blind. Seventeen minutes and thirty-two seconds, under budget: ~64.5k tokens against an 84k estimate. Codex counts tokens differently than Claude, so don’t compare the two runs’ absolutes. Four confirmed findings fixed, including untrimmed LOWER TRIM ... lookups and a backfiller shortcut that could link the wrong contact. Five claims refuted. 512 examples green, merged. And the run earned its keep by breaking something. The cold-reader’s brief named a file path, and in Codex the spawned agent had no file access . One wasted round. The fix made the canonical contract better for every tool: the document now travels embedded in the brief . That also hardens the experiment itself, since an embedded doc makes “read nothing else” physically true instead of politely requested. What didn’t work Not everything held up. Four things the runs pushed back on: - Scouts in your own repo are waste. They ran 3.3× their budget in trial 2, and I still had to read every core file to build. Scouts pay in foreign context only, and the playbook says so now. An empty phase isn’t a skipped one. A task with no foreign unknowns gets zero scouts, and that’s fine. What the no-skipping rule forbids is dropping a phase that has work. - The integration pass caught zero code defects across all runs. It earns its place catching spec/docs drift and packaging gaps, cheaply. But the verifiers do the hunting. - Codex’s read-only sandbox went empirically untested. Ironically, the prose rule worked too well for the agent to even attempt a probe. - The pipeline broke its own test-first rule once. It admitted it in the closing report, and the mutation checks caught the gap later. Honest reporting is a feature you have to demand explicitly, so the contracts demand it. Exit interviews After the runs I asked both agents the same open question, separately: how did the pipeline compare to your normal flow? Two caveats before you read them. Model quotes aren’t testimony, and I chose the excerpts. Their value is narrower and better: every factual claim in them is checkable against the run data above. And they converged without seeing each other’s answers. The Claude agent, on the reeve feature: “By the end of the build I thought I was done: tests green, rubocop clean. Then the review verifier found that two rotations with no write between them made the ledger read BROKEN, permanently. I had written that rule on purpose and believed it. … In my normal flow I’d have shipped all three.” The habit it said it would carry back, even without the pipeline: “verifiers who never saw my reasoning, given claims to break rather than code to admire.” The Codex agent: “I’d choose it for cross-cutting or risky changes where independent review matters. For routine work, the ceremony and token cost would outweigh the benefit.” They also agreed on the price. The Claude agent estimated the pipeline cost roughly 3× a direct build for that feature. That’s a guess, not a measurement, but an honest one. And it judged the trade worth it for that class of change : irreversible data, a security guarantee, a public-API version bump. That’s the scope statement, and the agents wrote it better than I could have: the pipeline is for changes where being wrong is expensive, not for the daily small stuff. A task too small for it is a task you do directly. The playbook is explicit about this: skip the whole ceremony then, never individual phases. One principle emerged that I haven’t seen named elsewhere: context locality . The framesmith bugs were fixed by framesmith’s agent, from an issue. The reeve feature was built by a session in reeve’s repo. My orchestrating session never carried another repo’s context. Writes happen where the context lives. What crosses boundaries is an artifact: a spec, a brief, or an issue complete enough to stand alone. It’s the single-writer rule, applied at the repo level. banneret The kit is extracted and open source: github.com/vicmaster/banneret https://github.com/vicmaster/banneret . It ships canonical contracts for scout, verifier and cold-reader, each defined by its artifacts brief in, report out and never by a runtime. It also ships the pipeline playbook. You compile both to whatever you use: npx banneret compile --target claude .claude/agents/ + /pipeline command npx banneret compile --target codex .codex/agents/ .toml + a managed AGENTS.md block The name: a banneret was a medieval knight who led a small company under his own banner. That’s the shape of the thing. Your session leads, and a small retinue reads and verifies. In Claude Code, /pipeline