Originally published on tamiz.pro.
The era of "vibe coding"—where developers rely on large language models (LLMs) to generate entire applications through natural language prompts without deep structural oversight—is hitting a hard ceiling. While LLMs are exceptional at generating boilerplate, writing unit tests, or refactoring legacy code, they are fundamentally probabilistic engines. They lack deterministic state management, persistent memory, and hard safety constraints.
As we move from prototyping to production-grade AI integration, the architecture of AI agents must evolve. We can no longer treat the LLM as a magic black box. Instead, we must engineer resilient AI agents by combining the creativity of generative models with the rigor of traditional software engineering. This article explores the triad of modern AI engineering: Finite State Machines (FSMs) for deterministic control flow, Local Privacy Layers for data security, and Cost Controls for economic viability.
In a typical "vibe-coded" scenario, an agent might be given a system prompt like:
system_prompt = "You are a helpful assistant. Help the user book a flight. If they mention a destination, search for flights. If they mention dates, book them."
This approach fails in production for three critical reasons:
To build resilient agents, we must wrap the LLM in a deterministic architecture. This is where Finite State Machines, local processing, and strict cost controls come into play.
An FSM is a mathematical model of computation. It consists of a finite number of states, transitions between those states, and actions. In the context of AI agents, the FSM acts as the "brain" or orchestrator, while the LLM acts as the "sensory organ" that interprets user intent and generates responses.
Let’s build a simple "Flight Booking Agent" using an FSM. We’ll use a library like transitions
or implement a lightweight state machine manually.
from enum import Enum
import json
class BookingState(Enum):
INITIAL = "initial"
COLLECTING_DESTINATION = "collecting_destination"
COLLECTING_DATES = "collecting_dates"
SEARCHING_FLIGHTS = "searching_flights"
CONFIRMING_BOOKING = "confirming_booking"
COMPLETED = "completed"
FAILED = "failed"
class FlightBookingAgent:
def __init__(self):
self.state = BookingState.INITIAL
self.context = {
"destination": None,
"departure_date": None,
"return_date": None
}
def handle_user_input(self, user_input: str) -> str:
intent_response = self._llm_interpret(user_input)
if self.state == BookingState.INITIAL:
self.state = BookingState.COLLECTING_DESTINATION
return "Where would you like to fly?"
elif self.state == BookingState.COLLECTING_DESTINATION:
self.context["destination"] = intent_response.get("destination")
if self.context["destination"]:
self.state = BookingState.COLLECTING_DATES
return "What dates do you want to travel?"
else:
return "I didn't understand the destination. Please try again."
elif self.state == BookingState.COLLECTING_DATES:
self.context["departure_date"] = intent_response.get("departure_date")
self.context["return_date"] = intent_response.get("return_date")
if self.context["departure_date"] and self.context["return_date"]:
self.state = BookingState.SEARCHING_FLIGHTS
return "Searching for flights..."
else:
return "I need both departure and return dates."
elif self.state == BookingState.SEARCHING_FLIGHTS:
self.state = BookingState.CONFIRMING_BOOKING
return "Found flight AA123. Book it?"
elif self.state == BookingState.CONFIRMING_BOOKING:
if intent_response.get("confirm"):
self.state = BookingState.COMPLETED
return "Booking confirmed!"
else:
self.state = BookingState.SEARCHING_FLIGHTS
return "Okay, let's search again."
return "I'm sorry, I'm not sure how to proceed."
def _llm_interpret(self, user_input: str) -> dict:
return {
"destination": "New York" if "new york" in user_input.lower() else None,
"departure_date": "2023-12-01" if "dec 1" in user_input.lower() else None,
"return_date": "2023-12-10" if "dec 10" in user_input.lower() else None,
"confirm": "yes" in user_input.lower()
}
For more complex agents, consider these patterns:
Booking
superstate with substates for Domestic
and International
. This reduces complexity in large systems.SEARCHING_FLIGHTS
to CONFIRMING_BOOKING
if the search returned results.Sending user data to public LLM APIs is a major privacy risk. To build resilient agents, you must implement a Local Privacy Layer that ensures sensitive data never leaves your infrastructure. This involves three key strategies:
Before sending any data to the LLM, strip out PII (Personally Identifiable Information). Use libraries like Presidio or custom regex rules to detect and redact:
import re
def redact_pii(text: str) -> str:
email_pattern = r'[a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\.[a-zA-Z]{2,}'
phone_pattern = r'\b\d{3}[-.]?\d{3}[-.]?\d{4}\b'
text = re.sub(email_pattern, '[EMAIL_REDACTED]', text)
text = re.sub(phone_pattern, '[PHONE_REDACTED]', text)
return text
For maximum privacy, run the LLM locally using open-source models like Llama 3, Mistral, or Gemma. This can be done using frameworks like:
Running locally ensures that:
Not all tasks require local processing. Use a hybrid approach:
This ensures that only non-sensitive data is sent to the cloud, reducing privacy risks while maintaining performance.
LLM APIs can be expensive, especially for agents that make multiple calls per user interaction. Without strict cost controls, your application can quickly incur significant bills.
Limit the number of tokens sent to the LLM. This includes:
max_tokens
in your API call to prevent runaway responses.
response = client.chat.completions.create(
model="gpt-4",
messages=messages,
max_tokens=500, # Limit output length
temperature=0.7,
stop=["\n\n"] # Stop generating at specific tokens
)
Implement real-time monitoring to track:
Use tools like LangSmith or Arize Phoenix to visualize and monitor LLM usage.
Combining FSMs, local privacy, and cost controls creates a robust, production-ready AI agent. Here’s a high-level architecture:
Q: Can I use an FSM with non-deterministic LLM outputs?
A: Yes, but you must treat LLM outputs as untrusted data. The FSM should validate LLM outputs against expected schemas (e.g., JSON schemas) and handle errors gracefully. Never trust the LLM to make state transitions directly.
Q: How do I balance privacy with the need for large knowledge bases?
A: Use a hybrid approach. Keep sensitive data local and use local models for PII-sensitive tasks. For tasks requiring vast knowledge, use cloud models but ensure data is anonymized or aggregated before sending. Consider using RAG (Retrieval-Augmented Generation) with local vector stores to keep data on-premise.
Q: What happens if the LLM fails to respond within a timeout?
A: The FSM should have a timeout mechanism. If the LLM fails to respond, the agent can fall back to a default response, retry the request, or escalate to a human operator. This ensures the system remains resilient even when external services fail.
The future of AI engineering is not about relying solely on the generative capabilities of LLMs. It is about engineering resilience by combining probabilistic models with deterministic architectures. By implementing Finite State Machines for control flow, Local Privacy Layers for data security, and strict Cost Controls for economic viability, we can build AI agents that are reliable, secure, and scalable.
As the industry moves beyond "vibe coding," engineers who master this triad will be best positioned to build the next generation of production-grade AI applications. The LLM is a tool, not a replacement for sound software engineering principles.
For more insights on AI engineering and production-ready systems, visit Tamiz's Insights.