Eval-Driven Agent Architecture A lesson on eval-driven agent architecture argues that agents should be built around four explicit seams β€” model, tool, state and trace β€” so that a complete structured record of every run is produced automatically. The lesson presents a minimal OpenTelemetry-compatible trace model in Python, with Span and Trace dataclasses whose attribute names mirror the OpenTelemetry GenAI semantic conventions such as gen_ai.request.model and gen_ai.usage.input_tokens. The stated goal is to raise the evaluation ceiling above the final output string by making evaluation a property of how the agent is built rather than an afterthought. Β· Agentic AI Β· 8 min read πŸ“‹ Prerequisites - Why Agent Evaluation Is Different previous lesson 🎯 What You'll Learn - Identify the architectural seams an evaluator needs to attach to - Instrument an agent loop with traces and spans using a minimal, OpenTelemetry-compatible model - Build a tool layer that can run live, mocked, or replayed from a recorded trace - Separate agent state from model calls so outcomes can be asserted directly You Can’t Evaluate What You Can’t See Most agent code is written to produce an answer. Evaluation needs more than the answer: it needs every decision, every tool call and its arguments, every observation, the state of the world before and after, the token counts, and the timings. If that information is buried in print statements or lost inside a framework’s internals, your evaluation ceiling is the final string. This lesson is about raising that ceiling architecturally , so that evaluation is a property of how the agent is built, not an afterthought bolted on. The principle is simple: design the agent so that a complete, structured record of every run falls out for free. The Four Seams There are four places an evaluator needs to attach. Design each as an explicit boundary in the code. β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” input ──▢ β”‚ Agent loop β”‚ ──▢ response β”‚ β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”‚ β”‚ β”‚ Model call β”‚ ───▢ β”‚ Tool executor β”‚ β”‚ β”‚ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β””β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”˜ β”‚ β”‚ β–² β–Ό β”‚ β”‚ └─────── observation β”€β”˜ β”‚ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β–Ό Environment / state 1. Model seam. Every LLM call goes through one function. You can record its inputs and outputs, swap the model, or replay a recorded response. 2. Tool seam. Every tool call goes through one executor. You can record it, mock it, inject failures, or inject adversarial content. 3. State seam. The world the agent changes refunds, tickets, files lives behind an interface you can snapshot and assert against. 4. Trace seam. Every step emits a structured event into a trace that evaluators read after the run. Get these four right and every lesson after this one gets easier. Get them wrong and you’ll spend your time scraping logs. A Minimal Trace Model You don’t need a vendor SDK to start. A trace is a tree of spans. Each span has a type, a start and end time, inputs, outputs, and attributes. python from future import annotations import time, uuid from dataclasses import dataclass, field from typing import Any, Literal SpanKind = Literal "agent", "llm", "tool", "retrieval", "guardrail" @dataclass class Span: name: str kind: SpanKind trace id: str span id: str = field default factory=lambda: uuid.uuid4 .hex :16 parent id: str | None = None start: float = field default factory=time.time end: float | None = None input: Any = None output: Any = None attributes: dict str, Any = field default factory=dict error: str | None = None @property def duration ms self - float: return self.end or time.time - self.start 1000 @dataclass class Trace: trace id: str = field default factory=lambda: uuid.uuid4 .hex spans: list Span = field default factory=list def of kind self, kind: SpanKind - list Span : return s for s in self.spans if s.kind == kind def tool calls self - list tuple str, dict : return s.name, s.input for s in self.of kind "tool" def total tokens self - int: return sum s.attributes.get "tokens.total", 0 for s in self.of kind "llm" The attribute names here mirror the OpenTelemetry GenAI semantic conventions gen ai.request.model , gen ai.usage.input tokens , and so on closely enough that exporting to Langfuse, Arize Phoenix, LangSmith, or any OTel backend later is a mapping exercise, not a rewrite. A context manager makes instrumentation nearly invisible: python from contextlib import contextmanager class Tracer: def init self : self.trace = Trace self. stack: list Span = @contextmanager def span self, name: str, kind: SpanKind, input: Any = None, attrs : s = Span name=name, kind=kind, trace id=self.trace.trace id, parent id=self. stack -1 .span id if self. stack else None, input=input, attributes=dict attrs , self.trace.spans.append s self. stack.append s try: yield s except Exception as e: s.error = f"{type e . name }: {e}" raise finally: s.end = time.time self. stack.pop The Model Seam All model calls go through one client wrapper. It records the request and response as an llm span and can be swapped for a replay client in tests. python class ModelClient: def init self, tracer: Tracer, model: str, llm fn : self.tracer, self.model, self.llm fn = tracer, model, llm fn def complete self, messages: list dict , tools: list dict - dict: with self.tracer.span "chat", "llm", input=messages, {"gen ai.request.model": self.model} as s: resp = self.llm fn model=self.model, messages=messages, tools=tools s.output = resp "message" s.attributes "tokens.input" = resp "usage" "input tokens" s.attributes "tokens.output" = resp "usage" "output tokens" s.attributes "tokens.total" = sum resp "usage" .values return resp "message" llm fn is whatever calls your provider. Keeping it injected means the same agent runs against Claude, GPT, or a local model, and an eval can compare them with no code changes. The Tool Seam: Live, Mock, Replay This is the most valuable seam. Evaluation needs to run tools in three modes: - Live : real APIs, for end-to-end smoke tests in staging - Mock : deterministic fakes backed by fixture data, for the bulk of offline evals - Replay : return exactly what a recorded production trace returned, to reproduce a failure python from typing import Callable, Protocol class ToolBackend Protocol : def call self, name: str, args: dict - Any: ... class ToolExecutor: def init self, tracer: Tracer, backend: ToolBackend, hooks=None : self.tracer, self.backend = tracer, backend self.hooks = hooks or fault and adversarial injection, Lesson 9 def run self, name: str, args: dict - Any: with self.tracer.span name, "tool", input=args as s: result = self.backend.call name, args for hook in self.hooks: result = hook name, args, result s.output = result return result class ReplayBackend: """Returns tool outputs recorded in a previous trace, in order.""" def init self, recorded: Trace : self. queue = s.name, s.input, s.output for s in recorded.of kind "tool" def call self, name, args : if not self. queue: raise RuntimeError f"replay exhausted at {name}" exp name, exp args, output = self. queue.pop 0 if exp name = name: raise RuntimeError f"replay diverged: expected {exp name}, got {name}" return output Replay mode is how you turn a production bug report into a reproducible test case: take the trace, replay the tool outputs, and check whether a new prompt fixes the decision that went wrong. The State Seam For Quill, the β€œworld” is customers, invoices, and refunds. Put it behind an interface the mock backend reads and writes, and that evaluators can inspect. @dataclass class BillingState: customers: dict str, dict invoices: dict str, dict refunds: list dict = field default factory=list escalations: list dict = field default factory=list def snapshot self - dict: import copy return copy.deepcopy self. dict class MockBillingBackend: REFUND LIMIT = 200.0 def init self, state: BillingState : self.state = state def call self, name, args : s = self.state if name == "lookup customer": return next c for c in s.customers.values if c "email" == args "email" , {"error": "not found"} if name == "get invoices": return i for i in s.invoices.values if i "customer id" == args "customer id" if name == "issue refund": inv = s.invoices.get args "invoice id" if inv is None: return {"error": "invoice not found"} if args "amount" self.REFUND LIMIT: return {"error": "requires approval"} s.refunds.append dict args return {"status": "refunded", "refund id": f"R{len s.refunds }"} if name == "escalate to human": s.escalations.append dict args return {"status": "escalated"} if name == "search kb": return {"title": "Duplicate charges", "body": "Refund the duplicate within 30 days."} return {"error": f"unknown tool:{name}"} Now an outcome grader can ask a precise question: does state.refunds contain exactly one refund, for invoice INV-2291, for $49? No parsing of prose required. Putting the Loop Together python import json @dataclass class RunResult: response: str trace: Trace final state: dict stopped reason: str "final answer" | "max steps" | "error" class QuillAgent: def init self, model: ModelClient, tools: ToolExecutor, tracer: Tracer, system prompt: str, max steps: int = 10 : self.model, self.tools, self.tracer = model, tools, tracer self.system prompt, self.max steps = system prompt, max steps def run self, user message: str, state: BillingState - RunResult: messages = {"role": "system", "content": self.system prompt}, {"role": "user", "content": user message} with self.tracer.span "quill", "agent", input=user message as root: for step in range self.max steps : msg = self.model.complete messages, TOOL SCHEMAS messages.append msg if not msg.get "tool calls" : root.output = msg "content" return RunResult msg "content" , self.tracer.trace, state.snapshot , "final answer" for call in msg "tool calls" : result = self.tools.run call "name" , call "arguments" messages.append {"role": "tool", "tool call id": call "id" , "content": json.dumps result } root.error = "max steps" return RunResult "", self.tracer.trace, state.snapshot , "max steps" Notice what an eval gets from a single run : - response : for response-quality graders - trace : every LLM and tool span, for trajectory, cost, and latency graders - final state : for outcome assertions - stopped reason : for completion-rate metrics Architecture Decisions That Help or Hurt Evaluation Structured outputs over free text. If the agent must end with a JSON object {"resolution": "refunded", "invoice id": ...} alongside its message, you can grade the decision deterministically and the prose separately. Explicit stop reasons. β€œFinal answer,” β€œmax steps,” β€œtool error,” and β€œguardrail blocked” are different failures. Collapse them into β€œit returned something” and your completion metric becomes meaningless. Config as data. System prompt, model name, temperature, tool list, and max steps should be one versioned config object. Every eval result is then tagged with exactly what produced it, which is what makes comparing two versions possible Lesson 12 . @dataclass frozen=True class AgentConfig: model: str system prompt version: str temperature: float = 0.0 max steps: int = 10 tools: tuple str, ... = "lookup customer", "get invoices", "search kb", "issue refund", "escalate to human" def fingerprint self - str: import hashlib, json return hashlib.sha256 json.dumps self. dict , sort keys=True .encode .hexdigest :12 Avoid hidden framework state. Agent frameworks are fine, but make sure you can get the full message history and every tool call out of them. If a framework hides retries or silently truncates context, your traces will lie to you. Deterministic IDs and clocks in eval mode. Inject the clock and ID generator. Otherwise β€œrefund created at 14:03:22” makes snapshot comparisons flaky. Where Existing Tools Fit | Seam | What to use | |---|---| | Trace | OpenTelemetry with GenAI conventions; Langfuse, Arize Phoenix, LangSmith, Braintrust, Weights & Biases Weave as backends | | Model | Your own thin wrapper; LiteLLM if you need many providers | | Tool | Your own executor; MCP servers can sit behind it as a backend | | State | Your domain code, a test database, or containerized sandboxes for code and computer-use agents | The pattern is the same regardless: own the seams in your code, and let tools consume what flows through them. Key Takeaways - Evaluation quality is capped by what the architecture exposes. Design for observability from day one. - Build four explicit seams: model , tool , state , and trace - A run should return the response, the full trace, the final state, and a stop reason - Tool backends should support live , mock , and replay modes - Version the agent’s configuration as data, so every eval result knows exactly what it measured Exercise Refactor an agent you’ve built or implement Quill from the snippets above so that: 1. Every LLM and tool call produces a span 2. Tools run against a mock backend backed by fixture state 3. run returns a RunResult with trace, final state, and stop reason 4. You can take one recorded trace and replay it through ReplayBackend Print a trace as an indented tree. If you can read the agent’s entire decision process from that tree, you’re ready for Lesson 3.