How We Cut 70% of Multi-Agent Token Waste by Replacing Supervisor LLMs with Typed State Machines

A developer redesigned a multi-agent AI runtime by replacing a central supervisor LLM with a deterministic typed state machine, cutting token consumption by more than 70% and eliminating non-deterministic supervisor drift. In the new design, worker agents return schema-validated receipts with explicit transition keys instead of free-form prose, and constraints are encoded as code-level transition guards rather than long supervisor prompts. Raw LLM transcripts are sealed to persistent storage while only the receipt is passed forward to the state machine.

If you have built a multi-agent AI system over the past two years, you have almost certainly encountered what we call the Supervisor Tax . The pattern usually starts with clean intentions: you have 3–4 specialized subagents a researcher, an executor, an evaluator, and a reporter coordinated by a central "Supervisor" or "Router" LLM. The supervisor inspects intermediate outputs, decides who gets called next, evaluates task completion, and synthesizes the final response. In local testing with 2 steps, it works great. But once you deploy it against real workloads with flaky APIs, 40-step workflows, and messy user requests, three problems immediately emerge: Here is how we redesigned our agent runtime to cut 70%+ of token consumption and eliminate non-deterministic supervisor drift. LLMs are extraordinary at fuzzy cognitive translation: understanding ambiguous user intent, parsing unstructured tool output, and authoring code or summaries. They are remarkably inefficient and unreliable at finite state routing. ❌ Traditional Hierarchical Supervisor Every Step Re-evaluates Context User Request │ ▼ ┌──────────────┐ Raw Prompt + History │ Supervisor │ ──────────────────────────► Worker Agent 1 │ LLM │ ◄────────────────────────── Natural Language Output └──────────────┘ Balloons Context Window │ ▼ ┌──────────────┐ │ Supervisor │ ──────────────────────────► Worker Agent 2 │ LLM │ ◄────────────────────────── ... └──────────────┘ ──────────────────────────────────────────────────────────────────────────── ✅ Typed State Machine Zero-Token Deterministic Handoff User Request ──► Intent Classifier / Fast Model ──► { State: RESEARCH } │ ▼ Worker Agent 1 │ ▼ Emits Typed Receipt { status: "SUCCESS", ... } │ Deterministic Transition Rule │ ▼ { State: CODE EXEC } When you replace the supervisor LLM with a deterministic typed state machine e.g. using XState, a custom DAG, or a lightweight transition matrix , every agent step has an explicit contract: Instead of letting worker agents dump markdown or free-form prose back to a coordinator, every leaf agent must return a schema-validated receipt. // types/agent-receipt.ts export interface AgentReceipt<TResult = unknown { stepId: string; agentName: string; status: "COMPLETED" | "FAILED" | "NEEDS HUMAN" | "RETRYABLE ERROR"; durationMs: number; tokensConsumed: { inputTokens: number; outputTokens: number; }; // The actual verifiable payload result: TResult; // Deterministic transition key nextTrigger: string; // Cryptographic or verifiable hash of artifacts created artifactHashes: string ; } When the worker finishes, its raw LLM transcript is sealed into a persistent session log on disk or in object storage, and only the receipt is passed forward to the state machine. Instead of writing long supervisor prompts like "Please make sure you only run the executor once and check that the tests pass before finishing" , encode these constraints as code-level transition guards: js // workflow/agent-machine.ts import { createMachine } from "xstate"; export const buildPipeline = createMachine { id: "agentPipeline", initial: "plan", states: { plan: { on: { PLAN VALIDATED: "execute", PLAN REJECTED: "plan retry", }, }, execute: { on: { EXECUTION SUCCESS: "verify", EXECUTION TIMEOUT: "recover state", }, }, verify: { on: { TESTS PASSED: "finalize", TESTS FAILED: "repair", }, }, repair: { // Hard ceiling: max 3 repair attempts before escalating to human always: { target: "escalate human", guard: { context } = context.repairCount = 3 } , on: { REPAIR READY: "execute", }, }, finalize: { type: "final" }, escalate human: { type: "final" }, }, } ; execute to verify costs exactly 0 LLM tokens and executes in sub-millisecond CPU time. context.repairCount = 3 halts execution immediately. An LLM supervisor will often retry 15 times before running out of max tokens. repair agent only receives the test failure diff and the code file, not the entire 30,000-token historical transcript of planning and exploratory browsing. When we migrated our production agent workflows from LLM supervisor loops to deterministic typed state transitions, here is what our telemetry recorded across 500+ complex multi-step tasks: Save the LLMs for what they do best: creative synthesis, complex reasoning, messy parsing, and domain coding. For the control plane, coordination, routing, and ceilings—stick to the tools computer science gave us fifty years ago: deterministic state machines, typed schemas, and verifiable receipts. What architecture does your team use to prevent agent routing drift? Drop your experience or questions in the comments below