{"slug": "designing-deterministic-ai-agent-loops-architecture-verification-and-replay", "title": "Designing Deterministic AI Agent Loops: Architecture, Verification, and Replay State Machines", "summary": "A developer has designed a deterministic AI agent execution runtime in TypeScript that replaces the naive LLM agent loop with a Finite State Machine architecture built around four phases: Observe, Propose, Verify, and Commit (OPVC). The runtime uses Zod-typed tool contracts and a Verification Gate that validates model-proposed actions against schemas and business invariants before execution, with event-sourced journaling to enable replay and prevent runaway loops. The approach targets common production failures such as infinite retry loops, hallucinated tool arguments, and invariant violations that inflate API costs and erode user trust.", "body_md": "Most software engineers building with Large Language Models (LLMs) eventually hit the exact same wall: **the naive agent loop problem**.\n\nYou start with a straightforward loop:\n\nIn demos, this works brilliantly. In production, it breaks in infuriatingly subtle ways. The agent gets stuck in infinite loops repeating failing tool calls, hallucinates arguments when tool output schema changes slightly, or produces intermediate outputs that violate domain invariants—costing hundreds of dollars in API credits while degrading user trust.\n\nThe core issue is architectural: **we treat AI agents like deterministic functions, while executing their state mutations non-deterministically without real boundary validation.**\n\nTo move from fragile AI prototypes to mission-critical infrastructure, we must treat the agent runtime as a **Finite State Machine (FSM) with isolated verification gates and event-sourced replay capabilities**.\n\nIn this article, we'll design and build a deterministic, production-ready AI agent execution runtime in TypeScript, step by step.\n\nA common fix for agent unreliability is \"prompt-based self-correction\"—telling the LLM in the system prompt: *\"If your tool execution fails, reflect on your mistake and try again.\"*\n\nThis fails in production for three distinct reasons:\n\nInstead of relying on the LLM to govern its own control flow, we decouple execution into four deterministic phases: **Observe, Propose, Verify, Commit (OPVC)**.\n\n``` php\nflowchart TD\n    A[State Machine Engine] -->|1. Observe Current State| B[Context Assembly]\n    B -->|2. Propose Action| C[LLM Planner]\n    C -->|Candidate Transition| D{3. Verification Gate}\n    D -->|Invalid / Invariant Violations| E[Synthesize Error Event]\n    E -->|Inject Guidance| A\n    D -->|Valid Transition| F[4. Commit Execution]\n    F -->|Execute Tool / Mutate State| G[Append Event to Journal]\n    G --> A\n```\n\nLet's construct a type-safe runtime that enforces this architecture.\n\nFirst, we establish strict typed contracts for our execution journal and tool schemas using Zod and TypeScript.\n\n``` js\nimport { z } from \"zod\";\n\n// Representing immutable events in our agent's history\nexport type AgentEvent =\n  | { type: \"USER_INPUT\"; payload: string; timestamp: number }\n  | { type: \"ACTION_PROPOSED\"; tool: string; args: unknown; timestamp: number }\n  | { type: \"VERIFICATION_FAILED\"; error: string; timestamp: number }\n  | { type: \"ACTION_COMMITTED\"; tool: string; result: unknown; timestamp: number }\n  | { type: \"AGENT_HALTED\"; reason: string; timestamp: number };\n\nexport interface AgentState {\n  history: AgentEvent[];\n  status: \"IDLE\" | \"AWAITING_PROPOSAL\" | \"VERIFYING\" | \"EXECUTING\" | \"COMPLETED\" | \"FAILED\";\n  consecutiveFailures: number;\n  maxRetries: number;\n}\n\n// Definition for a verifiable tool\nexport interface VerifiableTool<TInput = any, TOutput = any> {\n  name: string;\n  description: string;\n  schema: z.ZodSchema<TInput>;\n  // Deterministic guard checking business logic beyond pure JSON schema validation\n  guard?: (input: TInput, history: AgentEvent[]) => { valid: boolean; reason?: string };\n  execute: (input: TInput) => Promise<TOutput>;\n}\n```\n\nThe **Verification Gate** acts as an intermediate firewall between the model's output and your downstream services.\n\n```\nexport class VerificationGate {\n  constructor(private tools: Map<string, VerifiableTool>) {}\n\n  public verify(\n    toolName: string,\n    rawArgs: unknown,\n    history: AgentEvent[]\n  ): { success: true; validatedArgs: any } | { success: false; error: string } {\n    const tool = this.tools.get(toolName);\n    if (!tool) {\n      return { success: false, error: `Tool '${toolName}' does not exist.` };\n    }\n\n    // 1. Schema / Type Validation\n    const parseResult = tool.schema.safeParse(rawArgs);\n    if (!parseResult.success) {\n      return {\n        success: false,\n        error: `Schema mismatch for tool '${toolName}': ${parseResult.error.message}`,\n      };\n    }\n\n    // 2. Business Invariant / Guard Validation\n    if (tool.guard) {\n      const guardResult = tool.guard(parseResult.data, history);\n      if (!guardResult.valid) {\n        return {\n          success: false,\n          error: `Invariant guard failed for '${toolName}': ${guardResult.reason}`,\n        };\n      }\n    }\n\n    return { success: true, validatedArgs: parseResult.data };\n  }\n}\n```\n\nNotice the secondary `guard` step: schema parsing verifies *structural soundness*, but guards verify *semantic invariants* (e.g., \"cannot withdraw an amount greater than the current balance calculated from state history\").\n\nNow we wire this into a deterministic loop runner that manages state transitions cleanly:\n\n```\nexport interface LLMProvider {\n  proposeAction(\n    history: AgentEvent[],\n    availableTools: Array<{ name: string; description: string }>\n  ): Promise<{ tool: string; args: unknown } | { finalAnswer: string }>;\n}\n\nexport class DeterministicAgentEngine {\n  private state: AgentState;\n  private tools: Map<string, VerifiableTool> = new Map();\n  private verifier: VerificationGate;\n\n  constructor(\n    private llm: LLMProvider,\n    tools: VerifiableTool[],\n    maxRetries = 3\n  ) {\n    tools.forEach((t) => this.tools.set(t.name, t));\n    this.verifier = new VerificationGate(this.tools);\n    this.state = {\n      history: [],\n      status: \"IDLE\",\n      consecutiveFailures: 0,\n      maxRetries,\n    };\n  }\n\n  public async run(userInput: string): Promise<string> {\n    this.appendEvent({ type: \"USER_INPUT\", payload: userInput, timestamp: Date.now() });\n    this.state.status = \"AWAITING_PROPOSAL\";\n\n    while (this.state.status !== \"COMPLETED\" && this.state.status !== \"FAILED\") {\n      if (this.state.consecutiveFailures >= this.state.maxRetries) {\n        this.state.status = \"FAILED\";\n        this.appendEvent({\n          type: \"AGENT_HALTED\",\n          reason: `Exceeded max consecutive verification failures (${this.state.maxRetries}).`,\n          timestamp: Date.now(),\n        });\n        throw new Error(`Agent halted: Too many invalid action attempts.`);\n      }\n\n      // PHASE 1 & 2: OBSERVE & PROPOSE\n      const toolDescriptions = Array.from(this.tools.values()).map((t) => ({\n        name: t.name,\n        description: t.description,\n      }));\n\n      const proposal = await this.llm.proposeAction(this.state.history, toolDescriptions);\n\n      if (\"finalAnswer\" in proposal) {\n        this.state.status = \"COMPLETED\";\n        return proposal.finalAnswer;\n      }\n\n      this.appendEvent({\n        type: \"ACTION_PROPOSED\",\n        tool: proposal.tool,\n        args: proposal.args,\n        timestamp: Date.now(),\n      });\n\n      // PHASE 3: VERIFY\n      this.state.status = \"VERIFYING\";\n      const verification = this.verifier.verify(\n        proposal.tool,\n        proposal.args,\n        this.state.history\n      );\n\n      if (!verification.success) {\n        this.state.consecutiveFailures++;\n        this.appendEvent({\n          type: \"VERIFICATION_FAILED\",\n          error: verification.error,\n          timestamp: Date.now(),\n        });\n        this.state.status = \"AWAITING_PROPOSAL\";\n        continue; // Loop back; LLM sees the explicit VERIFICATION_FAILED event\n      }\n\n      // PHASE 4: COMMIT\n      this.state.status = \"EXECUTING\";\n      this.state.consecutiveFailures = 0; // Reset counter on valid proposal\n      const targetTool = this.tools.get(proposal.tool)!;\n\n      try {\n        const result = await targetTool.execute(verification.validatedArgs);\n        this.appendEvent({\n          type: \"ACTION_COMMITTED\",\n          tool: proposal.tool,\n          result,\n          timestamp: Date.now(),\n        });\n        this.state.status = \"AWAITING_PROPOSAL\";\n      } catch (execError: any) {\n        this.appendEvent({\n          type: \"VERIFICATION_FAILED\",\n          error: `Tool execution runtime error: ${execError.message}`,\n          timestamp: Date.now(),\n        });\n        this.state.status = \"AWAITING_PROPOSAL\";\n      }\n    }\n\n    throw new Error(\"Engine terminated unexpectedly.\");\n  }\n\n  private appendEvent(event: AgentEvent) {\n    this.state.history.push(event);\n  }\n\n  public getJournal(): readonly AgentEvent[] {\n    return Object.freeze([...this.state.history]);\n  }\n}\n```\n\nOne of the biggest pain points in agent operations is debugging production errors. When an agent fails on step 7 of an enterprise workflow, how do you reproduce it locally when LLM responses are inherently stochastic?\n\nBecause our runtime uses an **event-sourced journal**, we can implement **Deterministic Replay Mocking**.\n\n```\nexport class ReplayLLMProvider implements LLMProvider {\n  private proposalPointer = 0;\n\n  constructor(private recordedJournal: AgentEvent[]) {}\n\n  async proposeAction(): Promise<{ tool: string; args: unknown } | { finalAnswer: string }> {\n    // Advance to next recorded PROPOSED action in the event log\n    while (this.proposalPointer < this.recordedJournal.length) {\n      const event = this.recordedJournal[this.proposalPointer++];\n      if (event.type === \"ACTION_PROPOSED\") {\n        return { tool: event.tool, args: event.args };\n      }\n    }\n    return { finalAnswer: \"Replay execution completed.\" };\n  }\n}\n```\n\nBy injecting a `ReplayLLMProvider` populated with production error logs, developers can **replay the exact state sequence locally**, step through custom verification guards, and inspect invariant failures without burning LLM tokens or making single API calls.\n\n| Feature | Naive Agent Loop | Verifiable OPVC Runtime | \n|---|---|---|\n| **Control Flow** | Controlled by LLM output string | Governed by explicit FSM state | \n| **Tool Execution** | Directly triggered from raw output | Enforced via Schema + Semantic Invariant Guards | \n| **Failure Handling** | Prompt concatenation (\"Try again\") | Explicit failure events & finite retry budget | \n| **Side Effect Isolation** | None (Immediate mutation) | Staged commit phase | \n| **Debuggability** | Low (non-reproducible) | High (Event Sourcing & Replay Engine) | \n\nWhile this architectural pattern solves reliability issues, it introduces explicit system trade-offs that staff engineers must balance:\n\n`VERIFICATION_FAILED` events to the state journal consumes token window space.`consecutiveFailures > 1`, compact redundant failure events into a single summary frame before passing context to the LLM planner.\nNever allow verification functions to alter database state or call mutation endpoints. Verification functions **must be pure functions** operating only on the proposed payload and historical state.\n\nIf the LLM makes an error and the verification error message is vague (e.g., `Invalid input`), the LLM will repeatedly propose variations of the exact same broken input.\n\n**Fix**: Ensure your `VerificationGate` returns *actionable structural guidance*:\n\n```\n// BAD ERROR RESPONSE\n\"Invalid argument for refund.\"\n\n// GOOD ERROR RESPONSE\n\"Invariant Failure: Parameter 'amount' ($150) exceeds maximum order total ($100). Re-evaluate refund limits.\"\n```\n\nUse explicit OPVC State Machine architectures when:\n\nSkip this complexity when:\n\nBy treating AI agents not as magic autonomous entities, but as **probabilistic components operating inside deterministic state machines**, we construct AI systems that scale reliably in production environments.", "url": "https://wpnews.pro/news/designing-deterministic-ai-agent-loops-architecture-verification-and-replay", "canonical_source": "https://dev.to/staffsystems-lab/designing-deterministic-ai-agent-loops-architecture-verification-and-replay-state-machines-42jf", "published_at": "2026-09-15 02:48:59+00:00", "updated_at": "2026-09-15 03:31:07.750986+00:00", "lang": "en", "topics": ["ai-agents", "large-language-models", "ai-infrastructure", "developer-tools", "mlops"], "entities": ["TypeScript", "Zod"], "alternates": {"html": "https://wpnews.pro/news/designing-deterministic-ai-agent-loops-architecture-verification-and-replay", "markdown": "https://wpnews.pro/news/designing-deterministic-ai-agent-loops-architecture-verification-and-replay.md", "text": "https://wpnews.pro/news/designing-deterministic-ai-agent-loops-architecture-verification-and-replay.txt", "jsonld": "https://wpnews.pro/news/designing-deterministic-ai-agent-loops-architecture-verification-and-replay.jsonld"}}