Building Reliable AI Agents: Lessons from Failed Bots, FSMs, and the Hidden Costs of Agentic Workflows

A developer argues that building reliable AI agents requires imposing structure from traditional computer science, specifically Finite State Machines (FSMs), to prevent failure modes like hallucination loops, infinite recursion, and unbounded costs. The post details how naive agentic designs often collapse under probabilistic LLM behavior, and advocates for FSMs to govern LLM decision-making within defined states and transitions.

Originally published on tamiz.pro. The initial promise of Generative AI was straightforward: automate knowledge work. Early implementations often looked like sophisticated chatbots—stateless, reactive, and driven by a single prompt-response cycle. But as the industry matured, the demand shifted from conversational interfaces to Agentic Workflows : systems that perceive, plan, act, and observe to achieve complex, multi-step goals. The transition from chatbot to agent is not merely an architectural upgrade; it is a fundamental shift in how we reason about software reliability. While the early wave of "agents" often collapsed under the weight of hallucination loops, infinite recursion, and unbounded cost, a new class of engineering patterns is emerging. By borrowing determinism from traditional computer science—specifically Finite State Machines FSMs and rigorous observability—we can build systems that are not just clever, but robust. This deep-dive explores the architectural pitfalls of naive agentic designs, the critical role of state management in preventing failure loops, and the often-overlooked "hidden costs" that determine whether an agent stays in production or gets archived. Most failed AI agent projects share a common origin story. They begin with a powerful Large Language Model LLM and a tool definition e.g., a Python function to query a database . The initial architecture is typically a simple loop: This pattern, often called ReAct Reasoning + Acting , is elegant in theory but brittle in practice. When we remove the guardrails of traditional software engineering, three catastrophic failure modes emerge: LLMs are probabilistic, not deterministic. If a tool call fails e.g., a 500 error from a database , a naive agent may interpret the error message as valid data or, worse, hallucinate a successful completion to satisfy the user. Without explicit state validation, the agent enters a loop of retrying the same invalid action, consuming tokens and failing to resolve the user's intent. In complex workflows, an agent might decide it needs more information, call a tool to get it, and then decide it still needs more information. Without a hard limit on steps or a terminal state, the agent can enter an infinite loop of tool calls. This is not just a user experience failure; it is a financial liability. An infinite loop of token generation can burn through a budget in minutes. Every step in an agentic workflow adds history to the conversation context. As the agent performs more steps, the context window fills up. When the window is full, the LLM must summarize or drop earlier turns. This loss of context can cause the agent to forget its original goal or the results of previous actions, leading to disjointed and incoherent behavior. To build reliable agents, we must impose structure on the chaos of probabilistic generation. The most effective pattern for this is the Finite State Machine FSM . An FSM is a mathematical model of computation consisting of a finite number of states, transitions between those states, and actions. In the context of AI agents, an FSM does not replace the LLM; it governs it. The LLM becomes a component within the state machine, responsible for determining the next transition based on the current state and input. This separation of concerns is critical: The Orchestrator pattern is a common implementation of FSMs for agents. In this model, a central controller the FSM manages the workflow. It maintains the current state and decides which sub-agent or tool to invoke next. The LLM is invoked only when necessary to make a decision or generate content. python from enum import Enum from typing import Optional, Dict, Any Define the states of the agent class AgentState Enum : INIT = "init" PLAN = "plan" EXECUTE = "execute" VALIDATE = "validate" COMPLETE = "complete" FAILED = "failed" class AgentFSM: def init self : self.state = AgentState.INIT self.context: Dict str, Any = {} def transition self, new state: AgentState : Enforce valid transitions valid transitions = { AgentState.INIT: AgentState.PLAN , AgentState.PLAN: AgentState.EXECUTE, AgentState.FAILED , AgentState.EXECUTE: AgentState.VALIDATE , AgentState.VALIDATE: AgentState.COMPLETE, AgentState.PLAN , Loop back if validation fails AgentState.COMPLETE: , AgentState.FAILED: } if new state not in valid transitions.get self.state, : raise ValueError f"Invalid transition from {self.state} to {new state}" self.state = new state self.on state change self.state def on state change self, state: AgentState : if state == AgentState.PLAN: self. plan step elif state == AgentState.EXECUTE: self. execute step elif state == AgentState.VALIDATE: self. validate step elif state == AgentState.COMPLETE: print "Agent completed successfully." elif state == AgentState.FAILED: print "Agent failed. Check logs." def plan step self : LLM is called here to generate a plan pass def execute step self : LLM is called here to decide tool usage pass def validate step self : Logic to check if the tool output is correct pass By using an FSM, we guarantee that the agent cannot enter an invalid state. We can also enforce step limits at the FSM level, preventing infinite loops regardless of what the LLM decides. When engineering teams evaluate the viability of an AI agent, they often focus on accuracy metrics e.g., "Does the agent answer correctly 90% of the time?" but neglect the operational costs. The "hidden costs" of agentic workflows are significant and can render a successful agent economically unviable. Each step in an agentic workflow involves sending a prompt to the LLM. If a workflow requires 10 steps, you are paying for 10 separate LLM calls. Furthermore, as the context window grows with each step, the input token count increases, driving up costs non-linearly. Mitigation: Use smaller, specialized models for routing and classification tasks. Reserve expensive, large context models for final synthesis. Implement aggressive context management, such as summarizing past interactions or using vector retrieval to fetch only relevant context. In a traditional software system, a failed request is a single API call. In an agentic system, a failed goal might involve multiple tool calls, each consuming tokens. If an agent fails after 5 steps, you have paid for 5 steps of computation for zero value. Mitigation: Implement early termination checks. After each tool execution, validate the output against a success criteria. If the criteria are not met, fail fast and return an error or ask for user clarification, rather than continuing to burn tokens. Debugging an agent is exponentially harder than debugging a script. You cannot simply print console.log and trace the execution flow because the LLM's decision-making process is opaque. When an agent fails, you need to know: Which state was it in? What was the prompt? What was the tool output? What was the LLM's reasoning? Mitigation: Invest heavily in observability from day one. Use tracing tools like LangSmith, Langfuse, or Arize Phoenix to log every state transition, tool call, and LLM response. Structure your logs to include the full context of the decision-making process. Building a reliable agent requires more than just an FSM. It requires a holistic approach to engineering. Here are key best practices for production deployment: For high-stakes actions e.g., deleting a database record, sending an email to a customer , never let the agent act autonomously. Use the FSM to transition to a PENDING APPROVAL state, where a human must explicitly approve the action. LLMs are notoriously bad at adhering to strict formats. To prevent parsing errors, use LLM providers that support structured output e.g., JSON mode in OpenAI, Pydantic validation in LangChain . Define strict schemas for all tool inputs and outputs. Ensure that all tools used by the agent are idempotent. If an agent retries a failed step, the tool should produce the same result without causing side effects e.g., creating duplicate records . This is crucial for recovery from transient errors. Every agent should have a fallback path. If the LLM cannot make a decision, or if the tool execution fails repeatedly, the system should gracefully degrade. This might mean returning a generic error message, escalating to a human, or providing a static, pre-defined answer. The era of "magic" AI is over. Building reliable AI agents is no longer about finding the perfect prompt; it is about engineering robust systems that can handle uncertainty. By integrating Finite State Machines to enforce deterministic logic, implementing rigorous observability to track hidden costs, and designing for failure from the start, we can move beyond fragile prototypes to production-grade agentic workflows. The future of AI engineering lies not in making the LLM smarter, but in making the system around it more reliable. As you build your next agent, remember: the LLM is just one component in a larger, deterministic machine. A: Yes. The FSM is an architectural pattern that runs in your application code e.g., Python, TypeScript . It interacts with the LLM via API calls, so it is provider-agnostic. You can swap OpenAI, Anthropic, or local models without changing the FSM logic. A: For workflows that take minutes or hours, use asynchronous task queues e.g., Celery, BullMQ and persist the FSM state to a database. This allows the system to recover from crashes and resume from the last state. A: It depends. Orchestrator patterns are generally more deterministic and easier to debug because there is a single source of truth for the workflow. Multi-agent patterns where agents talk to each other are more flexible but significantly harder to control and debug. Start with Orchestrator for most use cases.