AIArticle
Stop relying on fragile agent frameworks. Build resilient agentic systems using deterministic workflows, state preservation, and robust harness engineering.
It is remarkably easy to build an AI agent demo that works once on a curated happy path. It is brutally difficult to build an agentic system that survives its first week in production. When developers move from simple "Ask" patterns (basic Retrieval-Augmented Generation) to "Do" patterns—where models autonomously select tools, route queries, and execute multi-step plans—they quickly run into the harsh realities of non-determinism, token bloat, API rate limits, and cascading failures.
If you have ever watched a runaway agent loop burn through fifty dollars of LLM tokens in three minutes while accomplishing absolutely nothing, you know the problem. The industry is beginning to realize that agentic systems are not magic; they are distributed systems in disguise.
To build systems that fail gracefully and recover predictably, we must move away from heavy, opaque agent frameworks and instead apply rigorous software engineering disciplines. By analyzing real-world deployments—such as Bayer’s Preclinical Information Center (PRINCE) platform—and architectural best practices from industry leaders, we can map out a practical blueprint for "context engineering" and "harness engineering" that makes agentic AI safe for production.
Workflows vs. Agents: The Fallacy of Pure Autonomy #
The first step toward reliability is choosing the right level of autonomy. In their architectural guidelines, Anthropic draws a sharp distinction between two patterns:
Workflows: Systems where LLMs and tools are orchestrated through predefined, deterministic code paths.Agents: Systems where the LLM dynamically directs its own process, tool usage, and step-by-step execution.
Many developers jump straight to fully autonomous agents, assuming the model can figure out the optimal path. In production, this is often a liability. Pure autonomy introduces unpredictability, making debugging nearly impossible and testing a moving target.
Instead, the most successful enterprise implementations use a hybrid approach: deterministic workflows with localized agentic decision-making.
For example, Bayer’s PRINCE platform—developed with Thoughtworks to navigate decades of complex, unstructured preclinical drug safety reports—evolved from a simple metadata search to an "Agentic RAG" system. Rather than letting a single agent run wild over the data, PRINCE uses specialized, single-purpose agents (Researcher, Reflection, and Writer) routed through a structured, multi-step pipeline.
By keeping the macro-routing deterministic (e.g., Clarify Intent → Plan → Research → Reflect → Write), you constrain the state space. The LLM is only autonomous within its designated step, drastically reducing the chance of catastrophic failure.
flowchart TD
A[User Input] --> B[Clarify Intent & Route]
B --> C[Think & Plan Agent]
C --> D[Execute Tool / Action]
D --> E[Reflection & Validation Agent]
E -- Data Insufficient --> C
E -- Data Sufficient --> F[Writer Agent / Synthesis]
F --> G[Human-in-the-Loop Review]
G --> H[Final Output]
Harness Engineering: Scaffolding the Unpredictable #
If "context engineering" is about shaping what information a model receives, harness engineering is about building the physical scaffolding around the model to maintain control. A robust agentic harness consists of three core pillars: state persistence, tool boundaries, and validation loops.
1. State Persistence and Durable Orchestration
Because agentic tasks can take minutes, hours, or even days to execute, they cannot rely on in-memory state. If a container restarts or a network call fails mid-workflow, the system must not lose its progress or re-run expensive LLM steps.
As the team at Temporal points out, agents must be treated as stateful, fault-tolerant systems. Using a durable execution engine allows you to persist the agent's state, history, and variables automatically. If a step fails, the workflow sleeps, retries with exponential backoff, or alerts a human—without losing the context of the previous steps.
2. Strict Tool Boundaries and Sandboxing
Agents interact with the world through tools—whether querying a SQL database, searching a vector store like pgvector, or calling external APIs via the Model Context Protocol.
To prevent "agentic misalignment" (where a model fabricates data or executes destructive actions to achieve a goal), tools must have strict boundaries. A tool should be a simple, single-purpose function with rigid input validation. The agent should never write raw SQL or execute arbitrary code unless it is running in a highly sandboxed, ephemeral environment.
3. Reflection and Validation Loops
Never trust an agent's first draft. A reliable architecture includes a dedicated "Reflection Agent" or programmatic validation gate. In the PRINCE architecture, the Reflection Agent acts as a quality gate, evaluating whether the retrieved data is sufficient to answer the user's question before handing it off to the Writer Agent. If the data is lacking, it routes the workflow back to the planning phase to gather more context.
The Developer Angle: Implementing a Resilient Agentic Pattern #
Let’s translate these architectural concepts into code. Below is a simplified Python implementation of a resilient, stateful workflow harness. It avoids bloated frameworks, relying instead on standard language features to implement explicit error boundaries, state tracking, and a validation loop.
import time
from typing import Dict, Any, List
class WorkflowState:
def __init__(self, query: str):
self.query: str = query
self.plan: List[str] = []
self.collected_data: List[Dict[str, Any]] = []
self.steps_completed: int = 0
self.max_steps: int = 5
self.status: str = "PENDING"
self.error_log: List[str] = []
class ResilientAgentHarness:
def __init__(self, llm_client, tools: Dict[str, Any]):
self.llm = llm_client
self.tools = tools
def execute(self, query: str) -> Dict[str, Any]:
state = WorkflowState(query)
state.plan = self._call_planner(state.query)
state.status = "RUNNING"
while state.steps_completed < state.max_steps:
try:
if self._is_task_complete(state):
state.status = "COMPLETED"
break
next_action = self._get_next_action(state)
result = self._execute_tool_with_retry(next_action)
state.collected_data.append(result)
state.steps_completed += 1
except Exception as e:
state.error_log.append(f"Step {state.steps_completed} failed: {str(e)}")
if not self._attempt_recovery(state, e):
state.status = "FAILED"
break
if state.status == "COMPLETED":
is_valid, feedback = self._validate_results(state)
if not is_valid:
state.error_log.append(f"Validation failed: {feedback}")
state.status = "PARTIAL_SUCCESS"
return {
"status": state.status,
"data": state.collected_data,
"errors": state.error_log
}
def _execute_tool_with_retry(self, action: Dict[str, Any], retries=3) -> Dict[str, Any]:
tool_name = action.get("tool")
tool_args = action.get("args", {})
if tool_name not in self.tools:
raise ValueError(f"Unauthorized tool: {tool_name}")
for attempt in range(retries):
try:
return self.tools[tool_name](**tool_args)
except Exception as e:
if attempt == retries - 1:
raise e
time.sleep(2 ** attempt) # Exponential backoff
def _call_planner(self, query: str) -> List[str]:
return ["search_database", "validate_results"]
def _get_next_action(self, state: WorkflowState) -> Dict[str, Any]:
return {"tool": "search_database", "args": {"query": state.query}}
def _is_task_complete(self, state: WorkflowState) -> bool:
return len(state.collected_data) > 0
def _attempt_recovery(self, state: WorkflowState, error: Exception) -> bool:
return True
def _validate_results(self, state: WorkflowState) -> tuple[bool, str]:
if not state.collected_data:
return False, "No data collected."
return True, "Success"
Trade-offs and Caveats
Implementing this level of scaffolding is not free. Developers must weigh several trade-offs:
Latency vs. Accuracy: Adding validation and reflection loops means executing multiple LLM calls sequentially. A single user query might take 15 seconds instead of 2. For real-time chat, this is painful; for asynchronous background tasks (like drafting regulatory documents in Bayer's case), it is entirely acceptable.Cost: More LLM calls mean higher token consumption. You must calculate whether the increased accuracy justifies the operational cost.Complexity: Writing custom state machines and retry logic requires more upfront engineering than importing a framework like LangChain or CrewAI. However, the payoff is a codebase that your team can actually debug, test, and maintain.
The Path Forward #
We are moving past the honeymoon phase of generative AI. Demos that rely on the model "just figuring it out" are being replaced by systems built on rigorous software engineering principles.
If you are building agentic systems today, stop looking for a magic framework to solve your reliability problems. Instead, focus on harness engineering: constrain your agents with deterministic workflows, enforce strict tool boundaries, persist state at every step, and build robust validation loops. Treat your agents like the unpredictable, distributed systems they are, and design them to fail gracefully from day one.
Sources & further reading #
Building reliable agentic AI systems— martinfowler.com - Best practices for building agentic systems | InfoWorld— infoworld.com - Building Effective AI Agents \ Anthropic— anthropic.com - Building an agentic system that’s actually production-ready | Temporal— temporal.io - Building Reliable Agentic AI Systems - geekfence.com— geekfence.com
Priya Nair· AI & Developer Experience Writer
Priya covers AI frameworks, developer productivity tooling, and the startup ecosystem across South and Southeast Asia, bringing a researcher's rigour and a practitioner's empathy to every story. She is deeply sceptical of benchmarks and asks hard questions so her readers don't have to.
Discussion 0 #
No comments yet
Be the first to weigh in.