# Why My AI Pipeline Has a Judge, Not Just Workers > Source: > Published: 2026-09-16 03:17:37+00:00 Most of what I ship today doesn't start with an editor open. It starts with a capable model reasoning through what the work actually requires, before any ticket exists. That model decides what it already knows and what it needs to go verify. Facts, current library versions, anything time-sensitive get handed to a cheap or free model whose only job is to go find them and report back. The reasoning itself never leaves the capable model. Once the answers come back, that same model writes the plan, the architecture, the stack, and only then breaks the work into tickets. Implementation is where the real division of labor happens. A ticket goes to a coding agent running on something cheap or fast. Review of that work goes to something stricter. These agents never talk to each other directly. Every result comes back to one orchestrator, and the orchestrator decides what happens next. A model that writes code doesn't get to also declare its own work correct, because that's a conflict of interest no harness should design around. A model that checks the work classifies what it finds and hands the classification back instead of quietly patching the problem itself. Four roles (personas) run in sequence once a ticket exists: a writer, a checker, an arbiter, and a final gate. The `writer` produces code, the `checker` as per its name states, tests it, the `arbiter` classifies any failure as real or environmental, and the final `gate` approves or sends it back. What moves between these roles is fixed. PASS or FAIL, written where a script can read it. Not a paragraph an agent has to interpret. Free-text handoffs fail quietly, which is the part that makes them dangerous. An agent can say a check "mostly" passes. It can hedge. It can restate the task instead of reporting a result. None of that gets caught at the handoff, so a human ends up resolving the same ambiguity anyway, just later and after more work has been built on top of it. The fixed contract removes that failure mode before it can start. If a report doesn't state PASS or FAIL in the field the script expects, the pipeline halts and asks rather than guessing. No agent trusts another agent's self-report; only the contract does. That's the actual line between "I used AI to go faster" and something a second person could pick up and run. I built a tool that lets an agent fetch a URL on my behalf, part of giving the research role something closer to real web access. An automated review caught a server-side request forgery risk in the first pass: nothing stopped the tool from being pointed at an internal address instead of a public one. I fixed it. A second review, run against my own fix, found the fix was incomplete. The blocklist checked the obvious private IP ranges but missed bracketed IPv6 loopback addresses and decimal-encoded IPv4 addresses, both of which resolve to the same internal targets through a different spelling. Neither agent that touched this code, the one that wrote it or the one that reviewed the first patch, caught what the second review found. I don't write validation logic assuming the first pass is complete anymore. I check the fix as hard as I checked the original feature, and I expect the same on the next one. Critical decisions stay mine on purpose, not because an agent couldn't attempt them but because **the cost of getting them wrong outweighs the time saved**. Ambiguous failures escalate instead of getting silently resolved. When something could be a real defect or could be an environment problem, that judgment call goes to a person, every time, no exceptions carved out for convenience. Scope gets written down before anyone starts, agent or human. I've had ambiguity resolved after the fact cost far more than the ten minutes it would have taken to write the scope down first, and I don't reopen that argument anymore. Several years (nine to be more factual) spent on legacy systems taught me what a clean migration looks like versus what a shortcut costs six months later. Running a pipeline of agents is that same judgment moved up a level: the review skill doesn't disappear when a model writes the first draft, it shifts to reviewing the process and designing the gates that catch what a first pass misses. The models got cheaper. The part where I decide what to trust them with didn't change at all.