How We Cut 70% of Multi-Agent Token Waste by Replacing Supervisor LLMs with Typed State Machines

If you have built a multi-agent AI system over the past two years, you have almost certainly encountered what we call the Supervisor Tax.

The pattern usually starts with clean intentions: you have 3–4 specialized subagents (a researcher, an executor, an evaluator, and a reporter) coordinated by a central "Supervisor" or "Router" LLM. The supervisor inspects intermediate outputs, decides who gets called next, evaluates task completion, and synthesizes the final response.

In local testing with 2 steps, it works great. But once you deploy it against real workloads with flaky APIs, 40-step workflows, and messy user requests, three problems immediately emerge:

Here is how we redesigned our agent runtime to cut 70%+ of token consumption and eliminate non-deterministic supervisor drift.

LLMs are extraordinary at fuzzy cognitive translation: understanding ambiguous user intent, parsing unstructured tool output, and authoring code or summaries.

They are remarkably inefficient and unreliable at finite state routing.

❌ Traditional Hierarchical Supervisor (Every Step Re-evaluates Context)
[User Request] 
      │
      ▼
┌──────────────┐    (Raw Prompt + History)
│  Supervisor  │ ──────────────────────────► [Worker Agent 1]
│     LLM      │ ◄────────────────────────── (Natural Language Output)
└──────────────┘    (Balloons Context Window)
      │
      ▼
┌──────────────┐
│  Supervisor  │ ──────────────────────────► [Worker Agent 2]
│     LLM      │ ◄────────────────────────── ...
└──────────────┘

────────────────────────────────────────────────────────────────────────────

✅ Typed State Machine (Zero-Token Deterministic Handoff)
[User Request] ──► [Intent Classifier / Fast Model] ──► { State: RESEARCH }
                                                               │
                                                               ▼
                                                     [Worker Agent 1]
                                                               │
                                                               ▼ (Emits Typed Receipt)
                                                     { status: "SUCCESS", ... }
                                                               │
                                     (Deterministic Transition Rule)
                                                               │
                                                               ▼
                                                     { State: CODE_EXEC }

When you replace the supervisor LLM with a deterministic typed state machine (e.g. using XState, a custom DAG, or a lightweight transition matrix), every agent step has an explicit contract:

Instead of letting worker agents dump markdown or free-form prose back to a coordinator, every leaf agent must return a schema-validated receipt.

// types/agent-receipt.ts
export interface AgentReceipt<TResult = unknown> {
  stepId: string;
  agentName: string;
  status: "COMPLETED" | "FAILED" | "NEEDS_HUMAN" | "RETRYABLE_ERROR";
  durationMs: number;
  tokensConsumed: {
    inputTokens: number;
    outputTokens: number;
  };
  // The actual verifiable payload
  result: TResult;
  // Deterministic transition key
  nextTrigger: string;
  // Cryptographic or verifiable hash of artifacts created
  artifactHashes: string[];
}

When the worker finishes, its raw LLM transcript is sealed into a persistent session log on disk or in object storage, and only the receipt is passed forward to the state machine.

Instead of writing long supervisor prompts like "Please make sure you only run the executor once and check that the tests pass before finishing", encode these constraints as code-level transition guards:

// workflow/agent-machine.ts
import { createMachine } from "xstate";

export const buildPipeline = createMachine({
  id: "agentPipeline",
  initial: "plan",
  states: {
    plan: {
      on: {
        PLAN_VALIDATED: "execute",
        PLAN_REJECTED: "plan_retry",
      },
    },
    execute: {
      on: {
        EXECUTION_SUCCESS: "verify",
        EXECUTION_TIMEOUT: "recover_state",
      },
    },
    verify: {
      on: {
        TESTS_PASSED: "finalize",
        TESTS_FAILED: "repair",
      },
    },
    repair: {
      // Hard ceiling: max 3 repair attempts before escalating to human
      always: [{ target: "escalate_human", guard: ({ context }) => context.repairCount >= 3 }],
      on: {
        REPAIR_READY: "execute",
      },
    },
    finalize: { type: "final" },
    escalate_human: { type: "final" },
  },
});

execute to verify costs exactly 0 LLM tokens and executes in sub-millisecond CPU time.context.repairCount >= 3 halts execution immediately. An LLM supervisor will often retry 15 times before running out of max tokens.repair agent only receives the test failure diff and the code file, not the entire 30,000-token historical transcript of planning and exploratory browsing. When we migrated our production agent workflows from LLM supervisor loops to deterministic typed state transitions, here is what our telemetry recorded across 500+ complex multi-step tasks:

Save the LLMs for what they do best: creative synthesis, complex reasoning, messy parsing, and domain coding.

For the control plane, coordination, routing, and ceilings—stick to the tools computer science gave us fifty years ago: deterministic state machines, typed schemas, and verifiable receipts.

What architecture does your team use to prevent agent routing drift? Drop your experience or questions in the comments below!