“Graph engineering” is about three weeks old as a term. It crystallized from a single tweet in mid-July, “Are we still talking loops or did we shift to graphs yet?”, and within days there were diagrams, Medium explainers, and a fresh rung on the prompt-engineering to context-engineering to loop-engineering ladder. What there wasn’t, as far as I can find, is a write-up from anyone who actually shipped something this way and published the numbers.
So here’s one.
Last week I built and released a Ruby gem, reeve, an authorization and audit layer for MCP servers. It gets its own introduction in a post shortly. I built it partly with parallel agents in isolated git worktrees, partly by hand, and reviewed the whole thing with a large agent fan-out. 68 tasks, 520 test examples, published to rubygems.org. The final review was a 37-agent fan-out.
The pitch you usually hear for this style of work is speed: n agents, roughly n times the throughput. That turned out to be the least interesting thing that happened.
The numbers #
| Phase | How | Tokens | Wall clock | Output |
|---|---|---|---|---|
| Kernel (6 classes, frozen) | Sequential, by hand | — | — | 148 examples |
| Module 1: authorization | Sequential, by hand | — | — | 86 examples |
| Module 2: audit ledger | 1 agent, isolated worktree | ~139k | ~17 min | 102 examples |
| Module 3: testing kit | 1 agent, isolated worktree | ~190k | ~21 min | 80 examples |
| Adversarial review | 37 agents, find then verify | ~1.5M | ~14 min | 10 defects, 9 confirmed |
Total agent spend was roughly 1.9M tokens against a 300k to 500k budget. Four times over. Every individual launch was estimated and approved in advance. Nobody was watching the running total. Hold that thought, it comes back at the end.
What worked #
The frozen interface was the whole trick
Before any agent started, I designed the kernel myself: six small classes (a Decision
that requires a rule string, a Context
, a configuration object, an error hierarchy, a ScopeResult
, an invocation envelope) plus four named collaborator protocols. Then I froze it under a git tag. The module agents built against fixed signatures. Neither touched a file the other needed. Neither guessed at an interface.
This is the boring precondition the buzzword skips: parallel agents don’t discover an architecture, they consume one. The kernel took longer to design than either module took to build, and that ratio is the story. If you can’t name your collaborator protocols before you fan out, you’re not ready to fan out.
Fan-out for verification beat fan-out for construction
The review used one finder agent per correctness angle, then an independent verifier per candidate finding. It cost more than everything else combined, and it was worth more than everything else combined. Nine confirmed defects, six of them silent fail-open paths in a library whose entire pitch is “deny by default, provably.”
Three of them, because the specifics are the point:
- A policy-resolution check used
`Object.const_defined?(name)`
, which returnstrue
forData
,Set
,File
,Range
, andProcess
. So a tool declaringguard_with DataPolicy
was silently governed by a different, convention-named policy. The guard the developer declared was not the guard that ran, and the audit ledger recorded the wrong one. Invisible from both sides. - When a policy scope couldn’t be read as a database relation (a Pundit
Scope#resolve
ending in.to_a
, a model whose primary key isn’tid
), the code abandoned the scope and re-authorized each record withshow?
, which in most codebases isuser.present?
. Every record kept. Recorded as properly scoped. Therescue
swallowed the cause. - The audit writer dropped the ledger row for every deniedcall when the write failed, because of a branch I’d written, carefully, to avoid masking the original exception. Denials are most of what an audit ledger is ever consulted about.
None of these crash. None fail a test you’d think to write. All of them fail open. They shipped fixed because 37 agents argued with the code, not because two agents wrote some of it fast.
The agents were more honest than I was
One module agent’s report volunteered that a task’s specs “never went red, that is a genuine test-first violation,” and that two checks ran against an injected fake rather than a genuinely broken system. Another flagged that it had guessed at an API that didn’t exist yet, and named the exact line that would need changing. That note is the only reason I caught the integration bug hiding there.
I made equivalent mistakes and noticed none of them until something forced the issue. An agent reporting into a review has no ego invested in the code it just wrote. Nobody lists this among the benefits of multi-agent work, and it might be the most reliable one.
What didn’t work #
Parallelism produces code, not integration
Both merges shipped defects that neither agent could have caught, because each side was correct in isolation. The audit module wired its redactor to a registry class while the authorization module exposed the registry as an instance, so per-tool redaction silently did nothing, with no error and no failing test on either side. And both modules wrote their own ActiveRecord test harness. Whichever ran second dropped the other’s schema, and sixteen tests went red on merge.
Every fan-out needs an integration pass owned by one mind, with a spec that exercises the seam. I nearly skipped it.
Distribution didn’t change the defect rate
Five of the ten review findings were in the module I wrote myself, sequentially, carefully, with full context. The agent-built modules had roughly the same defect density as mine. Parallelism bought wall-clock time. It did not buy correctness, and there’s no reason it should have. Anyone selling fan-out as a quality story is selling the wrong feature.
It didn’t save tokens either
The audit module’s ~139k tokens is roughly what it would have cost me to write interactively, plus I still had to read someone else’s work carefully afterward. The saving was wall-clock: 17 minutes of agent time for what would have occupied me an hour. That’s real if you have something else to do with the hour. It is not a cost saving.
The technique that mattered most isn’t graph-shaped #
After each of the ten review fixes, I reverted the fix and re-ran the suite to confirm it went red. Three of my regression specs turned out to be vacuous. One asserted behavior the test fixture produced anyway. One had a helper quietly re-declaring the policy under test, so my deliberately broken policy never ran. One covered a bug fixed in two independent code paths, so reverting either alone still passed.
Without the revert ritual, all three would have been recorded as coverage. Thirty seconds per fix, and the highest-value habit of the whole build.
The same discipline applies to reading agent output: a green report is a claim, not a result. The testing kit one agent built reported a clean pass against a deliberately leaky tool I wrote to check it. The kit was right, because a different bug of mine had made the leaky tool accidentally safe. Two wrongs looked like a pass.
Rules I’ll use next time #
Design and freeze the interface first, sequentially. If you can’t name the collaborator protocols, you’re not ready to fan out.Fan out generously for verification. For construction, only when modules are genuinely independent, and budget the review at least as much as the build.** Always budget an integration pass**, with a spec that crosses the seam. That’s where the bugs that survive both reviews live.** Ask every agent for its gaps explicitly, and read that section first.“Tell me what you left undone and which specs never went red” beat every quality metric I collected. Verify each fix by reverting it.This applies to your own work more than theirs. Track aggregate spend, not per-launch spend.**Four announced-and-approved launches still added up to 4x the budget. Individually reasonable decisions compound into an unreasonable one unless something is watching the total.
The one-line version #
Graph engineering is not a speed multiplier with a quality bonus. It’s an architecture-dependent wall-clock optimization with a verification multiplier attached, and the verification half is where the value actually is.
My gem shipped with six fewer silent security holes because thirty-seven agents argued with it. That’s the feature. The rest is diagrams.
reeve gets its own post soon. First I’m running it in a real app for a week to make sure it deserves one.