Designing Deterministic AI Agent Loops: Architecture, Verification, and Replay State Machines

A developer has designed a deterministic AI agent execution runtime in TypeScript that replaces the naive LLM agent loop with a Finite State Machine architecture built around four phases: Observe, Propose, Verify, and Commit (OPVC). The runtime uses Zod-typed tool contracts and a Verification Gate that validates model-proposed actions against schemas and business invariants before execution, with event-sourced journaling to enable replay and prevent runaway loops. The approach targets common production failures such as infinite retry loops, hallucinated tool arguments, and invariant violations that inflate API costs and erode user trust.

Most software engineers building with Large Language Models LLMs eventually hit the exact same wall: the naive agent loop problem . You start with a straightforward loop: In demos, this works brilliantly. In production, it breaks in infuriatingly subtle ways. The agent gets stuck in infinite loops repeating failing tool calls, hallucinates arguments when tool output schema changes slightly, or produces intermediate outputs that violate domain invariants—costing hundreds of dollars in API credits while degrading user trust. The core issue is architectural: we treat AI agents like deterministic functions, while executing their state mutations non-deterministically without real boundary validation. To move from fragile AI prototypes to mission-critical infrastructure, we must treat the agent runtime as a Finite State Machine FSM with isolated verification gates and event-sourced replay capabilities . In this article, we'll design and build a deterministic, production-ready AI agent execution runtime in TypeScript, step by step. A common fix for agent unreliability is "prompt-based self-correction"—telling the LLM in the system prompt: "If your tool execution fails, reflect on your mistake and try again." This fails in production for three distinct reasons: Instead of relying on the LLM to govern its own control flow, we decouple execution into four deterministic phases: Observe, Propose, Verify, Commit OPVC . php flowchart TD A State Machine Engine -- |1. Observe Current State| B Context Assembly B -- |2. Propose Action| C LLM Planner C -- |Candidate Transition| D{3. Verification Gate} D -- |Invalid / Invariant Violations| E Synthesize Error Event E -- |Inject Guidance| A D -- |Valid Transition| F 4. Commit Execution F -- |Execute Tool / Mutate State| G Append Event to Journal G -- A Let's construct a type-safe runtime that enforces this architecture. First, we establish strict typed contracts for our execution journal and tool schemas using Zod and TypeScript. js import { z } from "zod"; // Representing immutable events in our agent's history export type AgentEvent = | { type: "USER INPUT"; payload: string; timestamp: number } | { type: "ACTION PROPOSED"; tool: string; args: unknown; timestamp: number } | { type: "VERIFICATION FAILED"; error: string; timestamp: number } | { type: "ACTION COMMITTED"; tool: string; result: unknown; timestamp: number } | { type: "AGENT HALTED"; reason: string; timestamp: number }; export interface AgentState { history: AgentEvent ; status: "IDLE" | "AWAITING PROPOSAL" | "VERIFYING" | "EXECUTING" | "COMPLETED" | "FAILED"; consecutiveFailures: number; maxRetries: number; } // Definition for a verifiable tool export interface VerifiableTool<TInput = any, TOutput = any { name: string; description: string; schema: z.ZodSchema<TInput ; // Deterministic guard checking business logic beyond pure JSON schema validation guard?: input: TInput, history: AgentEvent = { valid: boolean; reason?: string }; execute: input: TInput = Promise<TOutput ; } The Verification Gate acts as an intermediate firewall between the model's output and your downstream services. export class VerificationGate { constructor private tools: Map<string, VerifiableTool {} public verify toolName: string, rawArgs: unknown, history: AgentEvent : { success: true; validatedArgs: any } | { success: false; error: string } { const tool = this.tools.get toolName ; if tool { return { success: false, error: Tool '${toolName}' does not exist. }; } // 1. Schema / Type Validation const parseResult = tool.schema.safeParse rawArgs ; if parseResult.success { return { success: false, error: Schema mismatch for tool '${toolName}': ${parseResult.error.message} , }; } // 2. Business Invariant / Guard Validation if tool.guard { const guardResult = tool.guard parseResult.data, history ; if guardResult.valid { return { success: false, error: Invariant guard failed for '${toolName}': ${guardResult.reason} , }; } } return { success: true, validatedArgs: parseResult.data }; } } Notice the secondary guard step: schema parsing verifies structural soundness , but guards verify semantic invariants e.g., "cannot withdraw an amount greater than the current balance calculated from state history" . Now we wire this into a deterministic loop runner that manages state transitions cleanly: export interface LLMProvider { proposeAction history: AgentEvent , availableTools: Array<{ name: string; description: string } : Promise<{ tool: string; args: unknown } | { finalAnswer: string } ; } export class DeterministicAgentEngine { private state: AgentState; private tools: Map<string, VerifiableTool = new Map ; private verifier: VerificationGate; constructor private llm: LLMProvider, tools: VerifiableTool , maxRetries = 3 { tools.forEach t = this.tools.set t.name, t ; this.verifier = new VerificationGate this.tools ; this.state = { history: , status: "IDLE", consecutiveFailures: 0, maxRetries, }; } public async run userInput: string : Promise<string { this.appendEvent { type: "USER INPUT", payload: userInput, timestamp: Date.now } ; this.state.status = "AWAITING PROPOSAL"; while this.state.status == "COMPLETED" && this.state.status == "FAILED" { if this.state.consecutiveFailures = this.state.maxRetries { this.state.status = "FAILED"; this.appendEvent { type: "AGENT HALTED", reason: Exceeded max consecutive verification failures ${this.state.maxRetries} . , timestamp: Date.now , } ; throw new Error Agent halted: Too many invalid action attempts. ; } // PHASE 1 & 2: OBSERVE & PROPOSE const toolDescriptions = Array.from this.tools.values .map t = { name: t.name, description: t.description, } ; const proposal = await this.llm.proposeAction this.state.history, toolDescriptions ; if "finalAnswer" in proposal { this.state.status = "COMPLETED"; return proposal.finalAnswer; } this.appendEvent { type: "ACTION PROPOSED", tool: proposal.tool, args: proposal.args, timestamp: Date.now , } ; // PHASE 3: VERIFY this.state.status = "VERIFYING"; const verification = this.verifier.verify proposal.tool, proposal.args, this.state.history ; if verification.success { this.state.consecutiveFailures++; this.appendEvent { type: "VERIFICATION FAILED", error: verification.error, timestamp: Date.now , } ; this.state.status = "AWAITING PROPOSAL"; continue; // Loop back; LLM sees the explicit VERIFICATION FAILED event } // PHASE 4: COMMIT this.state.status = "EXECUTING"; this.state.consecutiveFailures = 0; // Reset counter on valid proposal const targetTool = this.tools.get proposal.tool ; try { const result = await targetTool.execute verification.validatedArgs ; this.appendEvent { type: "ACTION COMMITTED", tool: proposal.tool, result, timestamp: Date.now , } ; this.state.status = "AWAITING PROPOSAL"; } catch execError: any { this.appendEvent { type: "VERIFICATION FAILED", error: Tool execution runtime error: ${execError.message} , timestamp: Date.now , } ; this.state.status = "AWAITING PROPOSAL"; } } throw new Error "Engine terminated unexpectedly." ; } private appendEvent event: AgentEvent { this.state.history.push event ; } public getJournal : readonly AgentEvent { return Object.freeze ...this.state.history ; } } One of the biggest pain points in agent operations is debugging production errors. When an agent fails on step 7 of an enterprise workflow, how do you reproduce it locally when LLM responses are inherently stochastic? Because our runtime uses an event-sourced journal , we can implement Deterministic Replay Mocking . export class ReplayLLMProvider implements LLMProvider { private proposalPointer = 0; constructor private recordedJournal: AgentEvent {} async proposeAction : Promise<{ tool: string; args: unknown } | { finalAnswer: string } { // Advance to next recorded PROPOSED action in the event log while this.proposalPointer < this.recordedJournal.length { const event = this.recordedJournal this.proposalPointer++ ; if event.type === "ACTION PROPOSED" { return { tool: event.tool, args: event.args }; } } return { finalAnswer: "Replay execution completed." }; } } By injecting a ReplayLLMProvider populated with production error logs, developers can replay the exact state sequence locally , step through custom verification guards, and inspect invariant failures without burning LLM tokens or making single API calls. | Feature | Naive Agent Loop | Verifiable OPVC Runtime | |---|---|---| | Control Flow | Controlled by LLM output string | Governed by explicit FSM state | | Tool Execution | Directly triggered from raw output | Enforced via Schema + Semantic Invariant Guards | | Failure Handling | Prompt concatenation "Try again" | Explicit failure events & finite retry budget | | Side Effect Isolation | None Immediate mutation | Staged commit phase | | Debuggability | Low non-reproducible | High Event Sourcing & Replay Engine | While this architectural pattern solves reliability issues, it introduces explicit system trade-offs that staff engineers must balance: VERIFICATION FAILED events to the state journal consumes token window space. consecutiveFailures 1 , compact redundant failure events into a single summary frame before passing context to the LLM planner. Never allow verification functions to alter database state or call mutation endpoints. Verification functions must be pure functions operating only on the proposed payload and historical state. If the LLM makes an error and the verification error message is vague e.g., Invalid input , the LLM will repeatedly propose variations of the exact same broken input. Fix : Ensure your VerificationGate returns actionable structural guidance : // BAD ERROR RESPONSE "Invalid argument for refund." // GOOD ERROR RESPONSE "Invariant Failure: Parameter 'amount' $150 exceeds maximum order total $100 . Re-evaluate refund limits." Use explicit OPVC State Machine architectures when: Skip this complexity when: By treating AI agents not as magic autonomous entities, but as probabilistic components operating inside deterministic state machines , we construct AI systems that scale reliably in production environments.