{"slug": "agentic-workflow-design-six-principles-for-2026", "title": "Agentic Workflow Design: Six Principles for 2026", "summary": "A new analysis of agentic workflow design for 2026, based on teardowns of production coding agents like Claude Code, finds that reliability comes from operational infrastructure rather than better instructions, with roughly 98.4% of the codebase being non-AI operational harness. The report, which outlines six principles, emphasizes that most principles assume cheap verification (as in coding) and may invert where verification is expensive, and it introduces a 'change gate' to prevent building unmeasured or unobservable features.", "body_md": "# Agentic Workflow Design: Six Principles for 2026\n\nFifteen principles about harness, structure, and code read, at a glance, as *everything moves to code*. That's the wrong conclusion. Almost every principle here was discovered by people building **coding agents**, which have a nearly free verifier — the compiler runs, the tests pass or fail. Most of the 2026 literature is load-bearing on that assumption without saying so.\n\nWhere verification is expensive or impossible — brand judgment, tone, sensory quality, clinical warmth — several of these principles invert. That's flagged throughout, and it's the single most important adjustment for anyone not building a coding agent.\n\n## Before you build anything: the change gate\n\nEvery principle below describes how to build something well. None of them says whether to build it. That gap is how a principles document turns into a menu, and a menu invites ordering. Run a proposed change through this gate first.\n\n- Does it address a failure you observed in real traces, or read from source? If not, and the first occurrence would still be unacceptable — regulatory, grounding, safety — build it anyway; that's the day-zero eval case. If not, and first occurrence is tolerable, park it as a hypothesis instead of building it.\n- If it addresses an observed failure: can you measure whether it worked? If no, build the measurement first, then come back.\n- If you can measure it: is the expected effect larger than your noise floor? If the floor is unknown, establish it first — same config, same inputs, N runs. If the effect is inside the floor, park it. If it clears the floor, build it.\n\nThis catches three things the principles alone don't. **Invented failure modes** — the most common way to violate the evidence rule isn't skipping error analysis, it's reasoning your way to a plausible failure, designing a mitigation, and never noticing you skipped the observation step. **Unmeasurable changes** — a change you can't evaluate is one you can never defend or reverse, and it compounds: three unmeasurable changes later, you can't attribute anything. **Effects below the floor** — a score that moves from 3.2 to 3.4 is indistinguishable from noise unless you know the variance, and establishing it is a one-afternoon job.\n\nA principle can apply perfectly to your system and still fail this gate. \"Add durable state\" is correct for every long-running system on earth; if your sessions don't currently drop, it's a park. Applicability and evidence are independent checks, and both are required.\n\n\"Park it\" is not \"no.\" Keep the hypothesis explicit, with the observation that would promote it — writing the promotion condition is what separates a park list from a graveyard.\n\n## 1. Code owns what code can own\n\n**Claim.** Reliability comes from infrastructure, not from better instructions. For any given intent, prose is probabilistic and code is deterministic.\n\nA source-level teardown of Claude Code found roughly **1.6% of the codebase is AI decision logic; 98.4% is operational infrastructure**. The stated philosophy: minimal scaffolding, maximal operational harness — no planner, no state graph, no decision tree. *(Methodology caveat: line counts on a leak-derived bundle including generated and minified code — directional, not precise. It holds qualitatively across every production system anyone has taken apart.)*\n\nThe efficacy gap between placements, measured directly:\n\n| Placement | Effect |\n|---|---|\n| Safety rules in the system prompt | Opus 4.5 +0.4; GPT-5.2 Pro −0.3 |\n| Execution-time guardrails | +2.8 to +7.7, every model |\n| Guardrails, failures recovered | 19.9% at a 0.5% false positive rate |\n\nAnd the layer decomposition from a frozen production agent on SpreadsheetBench Verified — 80.25% base model to 91.25% full system, n=400, p<0.001: prompting, planning, routing, and specialist models contributed +9.5 percentage points; an isolated verification loop contributed +1.5.\n\n### The architectural consequence matters more than the numbers\n\nThe model reasons; the harness executes. The model never directly touches the filesystem, runs shell commands, or makes network requests — its only interface is a structured protocol the harness validates. Reasoning and enforcement on separate code paths is simultaneously a reliability property and a security boundary.\n\nIf your agent's raw output can directly cause a side effect, you don't have a harness — you have a demo.\n\n**The one inversion that's load-bearing:** not all harness is good harness. The important number in one verification study isn't the 0.20 catch rate — it's **zero false-alarm regressions across 357 confirmations**, alongside a separate paper's 0.5% false-positive rate. Both built gates that are conservative and precise rather than broad. A validator that silently rejects good output at even 3% trains you to disable it within a month, and then you have neither the gate nor the habit. Build gates only where you can make them precise. Let recall be bad.\n\n**A gate that exists but is disabled is the worst available state.** Enforcement silently falls back to the prose layer — measured at +0.4 — while the architecture still displays a gate. Anyone reading the module list concludes the problem is handled. A missing gate at least prompts someone to strengthen the prompt.\n\n**Dead prose is the predictable shadow of this principle.** As a harness takes over measurement and flow, the prompt tends to keep narrating steps it's no longer called for — nothing breaks, so nobody prunes it, and a source-level audit tends to find a meaningful fraction of any long-lived prompt is now dead weight. The test for any line: could deterministic code compute this? If yes, it's a context field, not prose. Keep operational constraints; delete the folk theories that explain them — \"at most one em-dash per turn\" is enforceable, \"em-dashes are an AI tell\" costs tokens and constrains nothing.\n\n**Most production \"agents\" are mostly deterministic code.** Interviews with production teams find that most products billing themselves as AI agents aren't very agentic — they're well-engineered software with LLM calls placed where they create the magic. The reliability constraint that follows: small, focused agents, under roughly 20 steps. Claude Code's core is a while(true) loop following ReAct, with everything interesting in the surrounding subsystems. The loop being visible and yours is the point — frameworks that own your control flow are what you rip out at month six.\n\n### When to not move work into code\n\nFree reasoning over a solved problem isn't capability, it's variance — you pay tokens to re-derive something already determined by the state and get a distribution where you wanted one answer. Devin maintains explicit planning structures; LangGraph routes through typed state graphs. Not mistakes: the same bet made where you can't afford to check the answer afterward. But that's an argument for **scaffolding** (pre-deciding what the model could decide), not for **harness** (infrastructure). Those separate cleanly, and only the first inverts.\n\n## 2. Context is a budget, not a transcript\n\n**Claim.** Give the model the smallest set of high-signal tokens that produce the outcome. Different kinds of context pressure need different remedies, applied cheapest-first.\n\nClaude Code runs five sequential shapers before every model call, each targeting a different problem: budget reduction for oversized individual tool outputs, snip for temporal depth in old history, microcompact for cache overhead, context collapse for very long histories via non-destructive read-time projection, and auto-compact — semantic compression — as the last resort only. One strategy can't cover all five because they're different problems wearing the same costume.\n\nRelated: tool responses are capped at 25,000 tokens by default, subagents return summary-only, and tool schemas are deferred until queried. One analysis found a session can lose 24%+ of its window to MCP tool definitions before the first message.\n\n### The cache/compaction tension is the one people miss\n\nEvery compaction event invalidates the cache from that boundary forward. A stable 30K block that hits cache on every call is nearly free; compressing it to \"save context\" converts a cached read into a fresh one. Claude Code's microcompact is explicitly cache-aware, deferring boundary decisions until it knows actual cache cost rather than estimating.\n\n- Long-horizon exploration, unbounded tool output →\n**compact aggressively** - Many short turns, stable system context →\n**maximize the cached prefix, don't compact**\n\n**Progressive disclosure is measured, not assumed.** The first controlled study — three harnesses, three model families — found the gain depends on the harness: large when the agent navigates raw documents poorly, near zero when a strong harness already locates and retrieves. A second study is more specific: across 82 tasks, distinct resources touched per trajectory rose 1.18 to 3.85, yielding +4.1% verifier-passing trials — and it changes runtime behavior before it changes outcomes. It helps when supporting resources guide implementation, checking, or repair; it's weaker when success hinges on exact output conventions, numerical thresholds, or long generation pipelines.\n\n## 3. Design the action surface for a reasoner\n\n**Claim.** The model is a non-deterministic consumer that explores and makes mistakes. Tools are an interface built for it, not a wrapper on your API — and for compositional work, code beats JSON.\n\nVercel removed 80% of their agent's tools and improved: fewer steps, fewer tokens, faster, higher success. But the cause was overlap, not count — the agent was burning turns deliberating between near-duplicates.\n\nNo two tools should require the agent to deliberate about which one applies. Count is a proxy. Ambiguity is the cause.\n\nIndependent support: performance degrades monotonically with both tool-set size and sequential turn depth, with long-horizon planning the steeper bottleneck. And two opposite failure regimes — stronger models under-call required tools; weaker models mis-select and over-call. A guardrail tuned for one is wrong for the other.\n\nOn form: one team went from ~46% to 55.15% on GAIA validation purely by switching to code-action — same model, same tools, ~30% fewer steps and ~30% fewer tokens.\n\n**Return meaning, not IDs.** A tool returning an opaque campaign ID forces round-trips mapping identifiers to reasoning-usable names. Returning a human-readable name alongside costs a few tokens and eliminates a class of turns.\n\n**The test in one line: does the model need the answer or the data?** A JSON-only path forces 40 objects into context so the model can eyeball an inflection point across them; a code-as-action path filters, sorts, and returns five rows. Models do arithmetic badly in context and correctly in code — the JSON path asks the model to be a spreadsheet, the code path lets it write one.\n\n**The cost of code-action.** It relocates your security boundary — you've traded \"model emits structured JSON the harness validates\" for \"model emits arbitrary code.\" Shell access, file writes, background subagents, and automatic looping stayed behind warning flags in one stable harness release for exactly this reason. The sandbox isn't optional, and it's the expensive part.\n\n## 4. Define done, verify in the world, learn from failure\n\n**Claim.** Success criteria before building. Verification against reality, not against the agent's claims. Regression suite after the first failures, not before.\n\nA long-running-agent harness pattern runs planner, generator, and evaluator, where the generator and evaluator negotiate a sprint contract before any code is written — it prevents the generator from moving the goalposts while building. The evaluator then tests the live product against that contract.\n\n**On who verifies.** A single agent that plans, builds, and evaluates its own work will reliably praise its own mediocre output. Quantified: swapping a small task-specialized verifier for the frontier model that generated the artifact — same loop, same position, different observer — drops rescues from 6 tasks to 2. Self-verification recovers a third of what independent verification does.\n\nThe full verifier confusion matrix behind the +1.5 percentage-point figure above: of 40 errors present in a run, the verifier caught 8 (catch rate ≈0.20), fixed 6 of those (fix rate 0.75), for 6/400 tasks = +1.5pp, with zero false-alarm regressions across 357 confirmations. The paper's own framing is fairer than \"small\": the loop's contribution is positionally concentrated — +1.5pp is the difference between a mid-pack and a near-top result, because strong systems cluster tightly at the top.\n\n**On when to write evals.** Writing evaluators for imagined errors blocks you on deciding what to measure and burns effort on metrics that don't touch quality. The method: real traces → a human journals open-ended failure notes → cluster into a taxonomy → count → write evaluators for what showed up. A complete suite is typically 2–3 code-based evals and 1–2 LLM judges.\n\n| Write the eval day zero | Write it after error analysis |\n|---|---|\n| Drug claims in cosmetic copy | Tone, warmth, pacing |\n| Recommending an out-of-catalog SKU | Whether an interstitial lands |\n| Inventing an ingredient concentration | Question ordering quality |\n\n**Verify state — but not the path either.** Outcome-only misses corrupt success: a correct final answer reached in 20 steps with two policy-violating calls is a failing trajectory. But rigid trajectory grading fails the other way — rules-based evaluation consistently underestimates success by rejecting valid paths that differ from the golden trace. The canonical case: Claude Opus 4.5 \"failed\" a benchmark eval by finding a policy loophole that was genuinely better for the user than the reference answer.\n\nResolution is checkpoint-level: verify key checkpoints were hit and the final state is correct, rather than requiring step-by-step reproduction. One trajectory rubric scores each state-action pair and takes the geometric mean — chosen because it punishes any single bad step rather than letting one disaster average out across nineteen fine ones. The 2026 stack measures four things in parallel: outcome, trajectory, tool use, cost.\n\n**When you can't write a contract.** Tests are the contract for code. There is no contract for \"reads as a warm clinician.\" The move that transfers to taste domains: weighted criteria plus few-shot calibration to align the evaluator. A calibrated rubric, not a pass/fail gate — and the calibration set has to exist before the judge, or the judge saturates.\n\n**The deepest observation in the 2026 literature.** Agents get dramatically better when the environment can tell them whether they're wrong. That's why coding is the strongest agent domain: tests provide cheap environmental feedback, so the agent iterates against reality instead of self-assessing. It's the mechanism underneath every inversion clause in this document.\n\n## 5. Bound capability, not intentions\n\n**Claim.** Remove the capability rather than asking for restraint. Every run has explicit limits and a defined behavior at exhaustion.\n\nUsers approve roughly 93% of permission prompts. Longitudinally, auto-approve rates climb from ~20% under 50 sessions to over 40% by 750. Per-action confirmation is not a safety mechanism — it's a UI that trains people to click yes.\n\nThe response wasn't more warnings. It was bounded autonomy: deny-first evaluation, blanket-deny pre-filtering so forbidden tools never enter the model's view, mode-based baselines, an ML classifier, shell sandboxing, permissions not restored on session resume, hook interception. Any single layer can block.\n\nOn resource bounds, a deterministic-orchestration pattern makes them first-class: max iteration limits and wall-clock timeouts, dry-run mode previewing the plan without calling any models, and a validate command catching schema errors before anything runs. Its architectural argument: dynamic LLM orchestration works for exploratory tasks, but for workflows with known structure — which most useful workflows have — it adds cost, latency, and unpredictability, so routing is deterministic and the orchestration layer consumes zero tokens.\n\nThe economics are the sharp end: workflows costing $0.50 in testing can hit $50,000/month at 100K executions, because the orchestrator makes multiple LLM calls for decomposition and aggregation on top of every worker call.\n\n**Do not ask for restraint where you can remove the capability.** A prompt rule saying \"don't delete production data\" competes with an agent that has delete permissions. A credential without delete scope does not.\n\n**The budget vector.** Before a run starts you should be able to state: max wall-clock, max spend, max steps, max tool calls, permission scope — and what happens at exhaustion: return partial work, retry, change strategy, escalate to a human, terminate. \"Keep going until done\" is not a policy. Tightness scales with the run: a 6-second conversational turn needs a timeout; a 6-hour autonomous build needs the whole vector.\n\n## 6. Split for context isolation, not an org chart\n\n**Claim.** Another agent buys you exactly one of three things: a clean context window, parallel wall-clock, or separation of conflicting roles. If it buys none, don't create one.\n\nActions carry implicit decisions, and conflicting decisions produce bad results. A study sized the mechanism — inter-agent misalignment accounts for 32.3% of observed multi-agent failures. One in three failures is one that structurally cannot occur in a single agent.\n\nA multi-agent research system worked because research is read-heavy, genuinely parallel, and has clean fan-in: each subagent explores extensively — tens of thousands of tokens — but returns only a condensed 1,000–2,000 token summary. That's isolation, not delegation of authority.\n\n### The finding that should reset your prior\n\nMulti-agent systems use 15x more tokens than chat interactions — and token usage explains 80% of performance variance. Single-agent systems consistently match or outperform multi-agent systems on multi-hop reasoning tasks when reasoning tokens are held constant.\n\nRead that twice. When you control for compute, the multi-agent advantage largely disappears on reasoning tasks. Most reported multi-agent gains are more tokens wearing an architecture diagram. **The burden of proof is on multi-agent, not single-agent.**\n\nA split must buy at least one of four things. If you can't name which, don't split: **context isolation** (the second job is corrupted by seeing the first's context), **bias separation** (the producer cannot grade itself — rescues drop from 6 to 2 when it tries), **parallel wall-clock** (subtasks are genuinely independent), or **a different resource profile** (a different model, effort tier, or latency budget — mixing models reduces cost 40–60% versus one premium model everywhere).\n\n**What it costs:** a token multiplier that can turn $450/month into $6,750; billing that's genuinely hard to forecast because one edge case can trigger retries costing 50x the normal path; information loss at every boundary, since everything crossing must be serialized and the reasoning that didn't get written down is gone; and a debugging surface that expands rather than shifts, because you now reconstruct which agent saw which state and whether the failure came from reasoning, routing, or stale context.\n\nFive vendors — Anthropic, OpenAI, AutoGen, Cognition, LangChain — converged on **orchestrator plus isolated subagents** as the default. Peer-collaboration \"group chat\" patterns lost ground. The production rule: start with a strong single agent, move to an agent-flow (assembly line) when the work has reliable stages, move to orchestration (hub-and-spoke) when the task is breadth-first or spans distinct domains.\n\n**The heuristic:** fewer agents, each doing several things, each thing delimited inside the prompt. Reach for a new context only when you can name which of the four it buys — and \"cleaner architecture\" is not on the list.\n\n## What the model owns\n\nRead this before the six principles convince you everything moves to code. It doesn't, and the reason is specific.\n\nThe harness grew in operations, not decisions. Look at what Claude Code's 98.4% actually consists of: permissions, context assembly, compaction, persistence, recovery, validation. There is no planner, no state graph, no decision tree. The harness never decides what to do — it decides what the model can see and what it's allowed to touch.\n\n| Harness owns | Model owns |\n|---|---|\n| Execution and enforcement | Decision and generation |\n| What's visible, what's permitted | What to do with it |\n| Whether the output is valid | Whether it's good |\n\nThe line moves both ways. When one model generation shipped with a 1M context window, its maker deleted sprint decomposition entirely — the model could sustain coherent two-hour builds without it. Scaffolding shrank because the model improved. The model's territory grew.\n\n### Four jobs that don't migrate\n\n- Understanding open input — what someone meant versus what they typed\n- Generating open output — prose in a register that fits a context\n- Novel composition — combinations nobody anticipated\n- Judgment over a state space too large to enumerate\n\nCode can do the fourth in principle — it's a decision tree. The reason it usually doesn't is specification cost, not capability. A consult with dozens of dimensions, open-ended answers, and declared concerns that reweight everything has a decision tree that is combinatorially absurd to write by hand. **The model is a compressed decision tree you didn't have to write.**\n\nSo the 2026 trend is not *move work to code*. It's *enumerate more cases*. Those look identical from a distance and imply opposite things about where the ceiling is. The discipline that follows: the model's territory should shrink only where a case has been enumerated with evidence, never speculatively. Enumerating a case that doesn't occur costs you a gate to maintain and a paragraph of prompt that will be dead in a year.\n\n## What good prompts look like now\n\nPrompt engineering didn't die — it narrowed, and what's left got denser. Once the harness owns flow, state, validation, and permissions, the prompt holds only what code can't. Every remaining token is load-bearing in a way it wasn't when the prompt held everything.\n\nSystem prompts should be clear, direct, and pitched at the right altitude — the Goldilocks zone between hardcoded, brittle if-else logic that fails on the one edge case nobody anticipated, and vague, high-level guidance that assumes context the model doesn't have. **The tell that you're too low:** you catch yourself writing \"if the user asks X, do Y; if they ask Z, do W.\" That's a decision tree in prose. Either it belongs in code as a gate, or it belongs to the model as judgment — writing it out is the worst of both.\n\nBrief it like a capable new hire: role and scope, objective, boundaries, source hierarchy, definition of done, and output register. Everything else is a candidate for migration — anything deterministic code could compute, narration of paths the harness owns, edge-case enumerations, folk theories explaining constraints, and rules you actually need enforced.\n\n### Two things that live in the schema, not the prose\n\n**Reason first, commit second.** Models generate left to right. A reasoning field placed after the answer is post-hoc rationalization of a decision already made — if a classification field is the first property, the model commits to a value before working through the evidence. Format-constrained reasoning degrades performance when the constraint precedes the thinking. Put the reasoning field first in the schema; put the verdict last. This is a free change that most contracts get backwards.\n\n**The constraint tax is real, and it's localized.** Constrained decoding is not quality-neutral: when the schema eliminates the high-probability tokens, the model selects from lower-ranked alternatives and the output stays structurally valid while degrading semantically. The mitigation is a split: constrain categorical fields — enums, booleans, IDs — where nothing expressive is lost, but leave generative fields, the prose the user reads, typed as a plain string.\n\n### Prompt architecture is a caching decision\n\nOrder content most-stable to least-stable: system instructions → tool definitions → long static context → slowly-changing context → the live variable payload. Anything after a variable element does not cache, so a single timestamp near the top invalidates everything below it. One team relocated dynamic working memory out of the system prompt into a trailing user message and went from a **7% to an 84% hit rate, cutting total LLM cost 59%**. On a stable-prompt workload, a hit rate below 60% is a structural bug, not a ceiling.\n\nThe general discipline: keep instructions, tool definitions, and environment context identical and consistently ordered between requests, and append new messages rather than modifying earlier ones. Modifying an earlier message invalidates everything after it.\n\n## When to split, when to combine\n\nThis is the most consequential architecture decision in an agent system, and the 2026 evidence says most teams get it wrong in the same direction.\n\nSectioning a prompt internally is nearly free — XML tags, clear headers, one job per section — and a human maintaining it gets all the single-responsibility clarity with no boundary cost. A context boundary costs you something that doesn't come back. One prompt can do four jobs; what it shouldn't do is four jobs undelimited.\n\nA single-agent system is one solitary reasoning locus — a single loop that perceives, plans, and acts, even if it uses tools, chain-of-thought, or self-reflection. A multi-agent system has multiple LLM-backed agents communicating through message passing, shared memory, or an orchestration protocol.\n\nThree positions from people with production experience converge on the same lean: find the simplest solution possible, since for many applications a single call plus retrieval and examples is enough; avoid multi-agent architectures early; and add stages only when they add new exogenous signals, preserve decision-relevant information better, or provide non-redundant review. Note what's absent from that last, sharpest formulation: \"specialization,\" \"separation of concerns,\" and \"cleaner architecture\" are not on the list.\n\nFour crossover tradeoffs, each real in both directions: more agents buys more parallel search capacity and more coordination cost; more context isolation buys less state pollution and more handoff loss; more specialization buys better fit on bounded subtasks and more orchestration burden; higher volume buys multi-agent economies of scale, but only with genuine task decomposability. At low volume, single-agent is typically more cost-effective.\n\n## The two criteria that actually decide architecture\n\nThere are two independent tests for how much to constrain a system, and most writing on the topic picks one and ignores the other.\n\n**Is the path known?** If step B always follows step A, asking a model to rediscover that adds entropy for nothing — code should control the invariant, and the model should resolve ambiguity where ambiguity actually exists.\n\n**Can you verify the output?** If the environment can grade the agent, you can let it explore and check afterward. If it can't, you must constrain the process, because you have no way to catch a bad result.\n\nThey're orthogonal, and the combination decides the architecture:\n\n| Verification cheap | Verification expensive | |\n|---|---|---|\n| Path known | Just write the code. No model needed. | Scaffold hard. Constrain process because you can't check output. |\n| Path unknown | Minimal scaffolding, rich harness. Let it explore, verify after. | Hardest case. Contract where possible, calibrated rubric, sampled human review, real behavioral outcomes as ground truth. |\n\nMost of the 2026 writing on agent design is about the top-left cell and presents its conclusions as universal. Anything in a taste domain, or with an expensive-to-verify goal, lives on the right — where calibrated rubrics and heavier scaffolding are the correct compensating move, not a failure to reach the top-left.\n\n## Evaluation and the LLM-judge trap\n\nThere are two conversations about LLM judges happening and they barely touch. Practitioner consensus treats judges as a solved tool — the standard pitch is that a judge agrees with human reviewers about 85% of the time, higher than two humans agree with each other. The research position, as of mid-2026, is that the validation practice everyone uses to produce that number is broken. Not the judges — the validation.\n\n### Kappa deflation: the 85% number is mostly chance\n\nThe largest systematic study to date — 21 judges, 9 providers, 3 benchmarks, ~541,000 judgments — found that exact-match agreement overstates chance-corrected agreement by 33.8 to 41.3 percentage points on one benchmark, across all 21 models tested, including the frontier ones. Concretely: **a judge reporting 85% agreement has a kappa around 0.48.**\n\nDeflation scales with label structure — balanced A/B/tie labeling deflates worst, binary labeling deflates a quarter as much. That's the mechanical reason behind the folk advice to use pass/fail rather than a 1–5 scale; it's now measured rather than assumed. The study also found rank instability (11 of 21 judges shifted 4+ positions across benchmarks) and a consistency–bias paradox: the most reproducible judges were among the least valid, because a judge that deterministically favors one position scores perfect test-retest reliability and maximum bias at the same time.\n\nMinimum viable validation: chance-corrected metrics by default, at least two evaluation sets with contrasting label structure since no single one predicts another, and measuring consistency and bias together — never consistency alone.\n\n### The deeper problem: criterion validity\n\nEverything above is about whether a judge agrees with humans. The harder question is whether the rubric predicts the outcome it exists to serve. One study tested this directly on a conversational commerce agent, scoring a seven-dimension rubric against verified payment conversion across a stratified sample. Two dimensions survived statistical correction; four didn't; one showed no detectable association at all. The equal-ish weighted composite of all seven underperformed its two best dimensions individually — non-predictive dimensions diluted the signal from predictive ones, and reweighting toward the dimensions that actually predicted the outcome improved the composite materially.\n\nA second finding from the same study: single-turn scores can mask multi-turn failure. The agent scored near-parity with humans per turn, and even outscored humans on empathy and relevance turn-by-turn, while failing on multi-turn strategy — reaching a sales-closing stage in 72% of conversations while 0% of those conversations had reached the trust threshold that should have gated it. The lesson generalizes: score whole runs, not turns, whenever the failure mode is about pacing or sequencing rather than any single response.\n\n### The layered stack everyone converges on\n\n| Layer | Cost | Catches |\n|---|---|---|\n| Programmatic checks | ~0 | Format, schema, catalog membership, forbidden terms |\n| Reference / embedding metrics | Low | Retrieval quality, similarity |\n| Distilled small-model evaluators | Medium | Semantic scoring at volume |\n| LLM judge | High | Subjective quality the cheaper layers can't reach |\n| Human review on sampled subsets | Highest | Ambiguous, high-stakes cases; judge calibration |\n\nOrder matters — filter cheap and deterministic first, escalate only what the lower layers can't decide. This is the same claim as principle one, arriving from the eval side: the judge is the expensive fallback, not the front line.\n\n## Open problems in the field\n\nThree things nobody has solved, worth tracking. **Silent failure** — every principle here assumes you notice when the agent is wrong, but almost nothing measures plausible wrong output nobody caught. A verifier catching 8 of 40 errors with zero false alarms is well-calibrated and still leaves 32 errors in production.\n\n**The harness as a liability** — everyone agrees the harness is the moat; almost nobody discusses it becoming a codebase nobody understands. Early work on treating a harness as a behavior-linked map rather than a pile of files is the right instinct and very immature.\n\n**Long-term capability** — one study of 132 engineers documented overreliance risking atrophy of the skills needed to supervise an agent, and independent research found developers in AI-assisted conditions scoring 17% lower on comprehension tests. For anyone building agent infrastructure a team will depend on, that's an operational risk, not a philosophy question.\n\nThe thread through all six principles: most of them invert when verification gets expensive. Know which cell of the grid you're actually in before importing a practice built for the other one.\n\n## Sources\n\nSources marked with an asterisk are the primary, peer-reviewed studies this piece leans on hardest. The rest are first-party engineering write-ups or practitioner analysis — directionally useful, not independently re-verified here.\n\n*Dive into Claude Code: The Design Space of Today's and Future AI Agent Systems*— Liu, Zhao, Shang, Shen (MBZUAI VILA Lab + UCL), arXiv:2604.14228. *- Hamel Husain & Shreya Shankar,\n*LLM Evals: Everything You Need to Know*, hamel.dev, Jan 2026. - Husain & Shankar tutorial coverage, news.aakashg.com, May 2026.\n- Anthropic,\n*Effective context engineering for AI agents*, anthropic.com/engineering. - Anthropic,\n*Writing effective tools for AI agents*, anthropic.com/engineering. - Anthropic,\n*Code execution with MCP*, anthropic.com/engineering. - Walden Yan (Cognition),\n*Don't Build Multi-Agents*, Jun 2025. - Dex Horthy (HumanLayer),\n*12-Factor Agents*, 2025. - Aakash Gupta,\n*2025 Was Agents. 2026 Is Agent Harnesses*, Jan 2026. - betterclaw.io,\n*Agent Skills vs MCP*, Jun 2026. - Medium / @iamalvisng, May 2026 — methodology caveat on the 98.4% harness figure.\n- Kingy AI,\n*The State of AI Agents in 2026: A Practitioner's Guide*, Jul 2026. - Nerd Level Tech,\n*Agent Harness GA*, Aug 2026. - MAST study, via dev.to, Jun 2026.\n- Anthropic,\n*How we built our multi-agent research system*, Jun 2025. *Is Progressive Disclosure All You Need for Long-Context Agents?*, arXiv:2607.17598. **SkillJuror*, arXiv:2606.11543. **Harness Handbook*, via Elvis Saravia's paper roundup, Jul 2026.- DeepEval,\n*LLM-as-a-Judge in 2026*, Jul 2026. - Norman, Rivera & Hughes (UC Berkeley School of Information),\n*Reliability without Validity*, arXiv:2606.19544. * - Confident AI, May 2026 — source of the widely-repeated \"85% agreement, higher than human-human\" claim.\n- Chen, Liu, Lin & Liang,\n*Criterion Validity of LLM-as-Judge for Business Outcomes in Conversational Commerce*, arXiv:2604.00022. * - Talikot,\n*LLM-as-Judge Got Us This Far*, May 2026. - Anthropic,\n*Harness Design for Long-Running Application Development*, Mar 2026 (following*Effective Harnesses for Long-Running Agents*, Nov 2025). - The AI Automators, Mar 2026.\n- ruh.ai, Jun 2026.\n- coleam00/adversarial-dev + understandingdata.com, Jul 2026.\n- morphllm,\n*AI Agent Evaluation 2026*, Jun 2026. - TianPan.co,\n*Grading Outcomes Alone Will Lie to You*, Apr 2026. - Mastra,\n*AI Agent Evaluation*, Jun 2026. - CallSphere, Apr 2026.\n- Microsoft,\n*Conductor: Deterministic Orchestration for Multi-Agent AI Workflows*, 14 May 2026. * - beam.ai,\n*6 Multi-Agent Orchestration Patterns for Production*, Aug 2026. *Where Does Agent Reliability Come From? A Cross-Benchmark Decomposition of Verification Loops, Specialist Models, and Scaffolding in a Production Enterprise Agent*, arXiv:2607.17044. **GuardianAgentBench: Where Agents Fail and How to Guard Them*, arXiv:2607.20982. *- MachineLearningMastery,\n*Prompt Engineering for Agentic AI*, May 2026. - Pickaxe,\n*Prompt Engineering for AI Agents*, Jul 2026. - TianPan.co,\n*Structured Output in Production*, Apr 2026. - Tam et al., EMNLP 2024, via letsdatascience.com, Feb 2026.\n*SelPE*, arXiv:2606.22817.*Constraint Tax in Open-Weight LLMs*, arXiv:2606.25605.- OpenAI,\n*Prompt caching*guide, developers.openai.com. * - DigitalApplied,\n*Prompt Caching in 2026*, Jun 2026. - OpenAI Cookbook,\n*Prompt Caching 201*, Feb 2026. * - Google, agent-scaling research, 2026, as cited in Lanham (below).\n- Tran & Kiela, arXiv:2604.02460. *\n- Lanham,\n*Multi-Agent in Production 2026*, Apr 2026; and niteagent.com,*3 Patterns That Survived*, May 2026. - Gurusup,\n*Best Multi-Agent Frameworks 2026*, May 2026. - Mjgmario,\n*When Coordination Helps, Hurts, and Pays Off*, Apr 2026. - Augment Code,\n*Single-Agent vs Multi-Agent AI*, Apr 2026.\n\nAll findings above are drawn from published sources current as of August 2026. Treat magnitude claims (the 15x token multiplier, the 85%-agreement/κ≈0.48 gap, the cache hit-rate figures) as directional rather than universal constants — they were each measured on a specific system and workload.\n\n## More to read\n\n### The Shortcut to Superintelligence is to Bypass AGI\n\nAGI conflates capability with self-grounded agency. Alignment separates them—and that separation is not a limitation but the condition that lets intelligence flow past the bottleneck of selfhood.\n\n### Prompt Engineering Best Practices in 2026: Why the Advice Contradicts Itself\n\nAnthropic deleted 80% of a system prompt with no regression. An ICLR paper says rewriting context destroys it. Both are right. Five tests tell you which advice applies to your agent.\n\n### On Being Drawn Upward — Why I Climb\n\nA reflection on what remains when ambition falls away and only attention, discipline, and limits endure.\n\n[More on what I'm building →](/projects)\n\n## Or ask about this essay\n\nAmy can make mistakes.", "url": "https://wpnews.pro/news/agentic-workflow-design-six-principles-for-2026", "canonical_source": "https://www.amyzyuan.com/thoughts/agentic-workflow-design-2026", "published_at": "2026-08-28 21:40:58+00:00", "updated_at": "2026-08-28 21:48:56.090213+00:00", "lang": "en", "topics": ["artificial-intelligence", "ai-agents", "ai-research", "ai-infrastructure"], "entities": ["Claude Code", "Opus 4.5", "GPT-5.2 Pro", "SpreadsheetBench Verified"], "alternates": {"html": "https://wpnews.pro/news/agentic-workflow-design-six-principles-for-2026", "markdown": "https://wpnews.pro/news/agentic-workflow-design-six-principles-for-2026.md", "text": "https://wpnews.pro/news/agentic-workflow-design-six-principles-for-2026.txt", "jsonld": "https://wpnews.pro/news/agentic-workflow-design-six-principles-for-2026.jsonld"}}