# Designing Deterministic AI Agent Loops: Architecture, Verification, and Replay State Machines

> Source: <https://dev.to/staffsystems-lab/designing-deterministic-ai-agent-loops-architecture-verification-and-replay-state-machines-42jf>
> Published: 2026-09-15 02:48:59+00:00

Most software engineers building with Large Language Models (LLMs) eventually hit the exact same wall: **the naive agent loop problem**.

You start with a straightforward loop:

In demos, this works brilliantly. In production, it breaks in infuriatingly subtle ways. The agent gets stuck in infinite loops repeating failing tool calls, hallucinates arguments when tool output schema changes slightly, or produces intermediate outputs that violate domain invariants—costing hundreds of dollars in API credits while degrading user trust.

The core issue is architectural: **we treat AI agents like deterministic functions, while executing their state mutations non-deterministically without real boundary validation.**

To move from fragile AI prototypes to mission-critical infrastructure, we must treat the agent runtime as a **Finite State Machine (FSM) with isolated verification gates and event-sourced replay capabilities**.

In this article, we'll design and build a deterministic, production-ready AI agent execution runtime in TypeScript, step by step.

A common fix for agent unreliability is "prompt-based self-correction"—telling the LLM in the system prompt: *"If your tool execution fails, reflect on your mistake and try again."*

This fails in production for three distinct reasons:

Instead of relying on the LLM to govern its own control flow, we decouple execution into four deterministic phases: **Observe, Propose, Verify, Commit (OPVC)**.

``` php
flowchart TD
    A[State Machine Engine] -->|1. Observe Current State| B[Context Assembly]
    B -->|2. Propose Action| C[LLM Planner]
    C -->|Candidate Transition| D{3. Verification Gate}
    D -->|Invalid / Invariant Violations| E[Synthesize Error Event]
    E -->|Inject Guidance| A
    D -->|Valid Transition| F[4. Commit Execution]
    F -->|Execute Tool / Mutate State| G[Append Event to Journal]
    G --> A
```

Let's construct a type-safe runtime that enforces this architecture.

First, we establish strict typed contracts for our execution journal and tool schemas using Zod and TypeScript.

``` js
import { z } from "zod";

// Representing immutable events in our agent's history
export type AgentEvent =
  | { type: "USER_INPUT"; payload: string; timestamp: number }
  | { type: "ACTION_PROPOSED"; tool: string; args: unknown; timestamp: number }
  | { type: "VERIFICATION_FAILED"; error: string; timestamp: number }
  | { type: "ACTION_COMMITTED"; tool: string; result: unknown; timestamp: number }
  | { type: "AGENT_HALTED"; reason: string; timestamp: number };

export interface AgentState {
  history: AgentEvent[];
  status: "IDLE" | "AWAITING_PROPOSAL" | "VERIFYING" | "EXECUTING" | "COMPLETED" | "FAILED";
  consecutiveFailures: number;
  maxRetries: number;
}

// Definition for a verifiable tool
export interface VerifiableTool<TInput = any, TOutput = any> {
  name: string;
  description: string;
  schema: z.ZodSchema<TInput>;
  // Deterministic guard checking business logic beyond pure JSON schema validation
  guard?: (input: TInput, history: AgentEvent[]) => { valid: boolean; reason?: string };
  execute: (input: TInput) => Promise<TOutput>;
}
```

The **Verification Gate** acts as an intermediate firewall between the model's output and your downstream services.

```
export class VerificationGate {
  constructor(private tools: Map<string, VerifiableTool>) {}

  public verify(
    toolName: string,
    rawArgs: unknown,
    history: AgentEvent[]
  ): { success: true; validatedArgs: any } | { success: false; error: string } {
    const tool = this.tools.get(toolName);
    if (!tool) {
      return { success: false, error: `Tool '${toolName}' does not exist.` };
    }

    // 1. Schema / Type Validation
    const parseResult = tool.schema.safeParse(rawArgs);
    if (!parseResult.success) {
      return {
        success: false,
        error: `Schema mismatch for tool '${toolName}': ${parseResult.error.message}`,
      };
    }

    // 2. Business Invariant / Guard Validation
    if (tool.guard) {
      const guardResult = tool.guard(parseResult.data, history);
      if (!guardResult.valid) {
        return {
          success: false,
          error: `Invariant guard failed for '${toolName}': ${guardResult.reason}`,
        };
      }
    }

    return { success: true, validatedArgs: parseResult.data };
  }
}
```

Notice the secondary `guard` step: schema parsing verifies *structural soundness*, but guards verify *semantic invariants* (e.g., "cannot withdraw an amount greater than the current balance calculated from state history").

Now we wire this into a deterministic loop runner that manages state transitions cleanly:

```
export interface LLMProvider {
  proposeAction(
    history: AgentEvent[],
    availableTools: Array<{ name: string; description: string }>
  ): Promise<{ tool: string; args: unknown } | { finalAnswer: string }>;
}

export class DeterministicAgentEngine {
  private state: AgentState;
  private tools: Map<string, VerifiableTool> = new Map();
  private verifier: VerificationGate;

  constructor(
    private llm: LLMProvider,
    tools: VerifiableTool[],
    maxRetries = 3
  ) {
    tools.forEach((t) => this.tools.set(t.name, t));
    this.verifier = new VerificationGate(this.tools);
    this.state = {
      history: [],
      status: "IDLE",
      consecutiveFailures: 0,
      maxRetries,
    };
  }

  public async run(userInput: string): Promise<string> {
    this.appendEvent({ type: "USER_INPUT", payload: userInput, timestamp: Date.now() });
    this.state.status = "AWAITING_PROPOSAL";

    while (this.state.status !== "COMPLETED" && this.state.status !== "FAILED") {
      if (this.state.consecutiveFailures >= this.state.maxRetries) {
        this.state.status = "FAILED";
        this.appendEvent({
          type: "AGENT_HALTED",
          reason: `Exceeded max consecutive verification failures (${this.state.maxRetries}).`,
          timestamp: Date.now(),
        });
        throw new Error(`Agent halted: Too many invalid action attempts.`);
      }

      // PHASE 1 & 2: OBSERVE & PROPOSE
      const toolDescriptions = Array.from(this.tools.values()).map((t) => ({
        name: t.name,
        description: t.description,
      }));

      const proposal = await this.llm.proposeAction(this.state.history, toolDescriptions);

      if ("finalAnswer" in proposal) {
        this.state.status = "COMPLETED";
        return proposal.finalAnswer;
      }

      this.appendEvent({
        type: "ACTION_PROPOSED",
        tool: proposal.tool,
        args: proposal.args,
        timestamp: Date.now(),
      });

      // PHASE 3: VERIFY
      this.state.status = "VERIFYING";
      const verification = this.verifier.verify(
        proposal.tool,
        proposal.args,
        this.state.history
      );

      if (!verification.success) {
        this.state.consecutiveFailures++;
        this.appendEvent({
          type: "VERIFICATION_FAILED",
          error: verification.error,
          timestamp: Date.now(),
        });
        this.state.status = "AWAITING_PROPOSAL";
        continue; // Loop back; LLM sees the explicit VERIFICATION_FAILED event
      }

      // PHASE 4: COMMIT
      this.state.status = "EXECUTING";
      this.state.consecutiveFailures = 0; // Reset counter on valid proposal
      const targetTool = this.tools.get(proposal.tool)!;

      try {
        const result = await targetTool.execute(verification.validatedArgs);
        this.appendEvent({
          type: "ACTION_COMMITTED",
          tool: proposal.tool,
          result,
          timestamp: Date.now(),
        });
        this.state.status = "AWAITING_PROPOSAL";
      } catch (execError: any) {
        this.appendEvent({
          type: "VERIFICATION_FAILED",
          error: `Tool execution runtime error: ${execError.message}`,
          timestamp: Date.now(),
        });
        this.state.status = "AWAITING_PROPOSAL";
      }
    }

    throw new Error("Engine terminated unexpectedly.");
  }

  private appendEvent(event: AgentEvent) {
    this.state.history.push(event);
  }

  public getJournal(): readonly AgentEvent[] {
    return Object.freeze([...this.state.history]);
  }
}
```

One of the biggest pain points in agent operations is debugging production errors. When an agent fails on step 7 of an enterprise workflow, how do you reproduce it locally when LLM responses are inherently stochastic?

Because our runtime uses an **event-sourced journal**, we can implement **Deterministic Replay Mocking**.

```
export class ReplayLLMProvider implements LLMProvider {
  private proposalPointer = 0;

  constructor(private recordedJournal: AgentEvent[]) {}

  async proposeAction(): Promise<{ tool: string; args: unknown } | { finalAnswer: string }> {
    // Advance to next recorded PROPOSED action in the event log
    while (this.proposalPointer < this.recordedJournal.length) {
      const event = this.recordedJournal[this.proposalPointer++];
      if (event.type === "ACTION_PROPOSED") {
        return { tool: event.tool, args: event.args };
      }
    }
    return { finalAnswer: "Replay execution completed." };
  }
}
```

By injecting a `ReplayLLMProvider` populated with production error logs, developers can **replay the exact state sequence locally**, step through custom verification guards, and inspect invariant failures without burning LLM tokens or making single API calls.

| Feature | Naive Agent Loop | Verifiable OPVC Runtime | 
|---|---|---|
| **Control Flow** | Controlled by LLM output string | Governed by explicit FSM state | 
| **Tool Execution** | Directly triggered from raw output | Enforced via Schema + Semantic Invariant Guards | 
| **Failure Handling** | Prompt concatenation ("Try again") | Explicit failure events & finite retry budget | 
| **Side Effect Isolation** | None (Immediate mutation) | Staged commit phase | 
| **Debuggability** | Low (non-reproducible) | High (Event Sourcing & Replay Engine) | 

While this architectural pattern solves reliability issues, it introduces explicit system trade-offs that staff engineers must balance:

`VERIFICATION_FAILED` events to the state journal consumes token window space.`consecutiveFailures > 1`, compact redundant failure events into a single summary frame before passing context to the LLM planner.
Never allow verification functions to alter database state or call mutation endpoints. Verification functions **must be pure functions** operating only on the proposed payload and historical state.

If the LLM makes an error and the verification error message is vague (e.g., `Invalid input`), the LLM will repeatedly propose variations of the exact same broken input.

**Fix**: Ensure your `VerificationGate` returns *actionable structural guidance*:

```
// BAD ERROR RESPONSE
"Invalid argument for refund."

// GOOD ERROR RESPONSE
"Invariant Failure: Parameter 'amount' ($150) exceeds maximum order total ($100). Re-evaluate refund limits."
```

Use explicit OPVC State Machine architectures when:

Skip this complexity when:

By treating AI agents not as magic autonomous entities, but as **probabilistic components operating inside deterministic state machines**, we construct AI systems that scale reliably in production environments.