# Eval-Driven Agent Architecture

> Source: <https://superml.org/tutorials/agent-eval-driven-architecture>
> Published: 2026-11-18 00:00:00+00:00

**Recommendation:** Instrument the agent loop with a single model seam, tool executor, state interface and trace emitter so every run yields a structured span tree; keep tool calls replayable and mockable to test failures and adversarial input.
More in the playbook "Designing agent systems": <https://wpnews.pro/playbooks/agent-architecture>

· Agentic AI · 8 min read

### 📋 Prerequisites

- Why Agent Evaluation Is Different (previous lesson)

### 🎯 What You'll Learn

- Identify the architectural seams an evaluator needs to attach to
- Instrument an agent loop with traces and spans using a minimal, OpenTelemetry-compatible model
- Build a tool layer that can run live, mocked, or replayed from a recorded trace
- Separate agent state from model calls so outcomes can be asserted directly

## You Can’t Evaluate What You Can’t See

Most agent code is written to produce an answer. Evaluation needs more than the answer: it needs every decision, every tool call and its arguments, every observation, the state of the world before and after, the token counts, and the timings.

If that information is buried in `print` statements or lost inside a framework’s internals, your evaluation ceiling is the final string. This lesson is about raising that ceiling *architecturally*, so that evaluation is a property of how the agent is built, not an afterthought bolted on.

The principle is simple: **design the agent so that a complete, structured record of every run falls out for free.**

## The Four Seams

There are four places an evaluator needs to attach. Design each as an explicit boundary in the code.

```
            ┌──────────────────────────────────────────┐
  input ──▶ │  Agent loop                              │ ──▶ response
            │   ┌────────────┐      ┌───────────────┐  │
            │   │ Model call │ ───▶ │ Tool executor │  │
            │   └────────────┘      └───────┬───────┘  │
            │         ▲                     ▼          │
            │         └─────── observation ─┘          │
            └───────────────┬──────────────────────────┘
                            ▼
                   Environment / state
```

1. **Model seam.** Every LLM call goes through one function. You can record its inputs and outputs, swap the model, or replay a recorded response.
2. **Tool seam.** Every tool call goes through one executor. You can record it, mock it, inject failures, or inject adversarial content.
3. **State seam.** The world the agent changes (refunds, tickets, files) lives behind an interface you can snapshot and assert against.
4. **Trace seam.** Every step emits a structured event into a trace that evaluators read after the run.

Get these four right and every lesson after this one gets easier. Get them wrong and you’ll spend your time scraping logs.

## A Minimal Trace Model

You don’t need a vendor SDK to start. A trace is a tree of spans. Each span has a type, a start and end time, inputs, outputs, and attributes.

``` python
from __future__ import annotations
import time, uuid
from dataclasses import dataclass, field
from typing import Any, Literal

SpanKind = Literal["agent", "llm", "tool", "retrieval", "guardrail"]

@dataclass
class Span:
    name: str
    kind: SpanKind
    trace_id: str
    span_id: str = field(default_factory=lambda: uuid.uuid4().hex[:16])
    parent_id: str | None = None
    start: float = field(default_factory=time.time)
    end: float | None = None
    input: Any = None
    output: Any = None
    attributes: dict[str, Any] = field(default_factory=dict)
    error: str | None = None

    @property
    def duration_ms(self) -> float:
        return ((self.end or time.time()) - self.start) * 1000

@dataclass
class Trace:
    trace_id: str = field(default_factory=lambda: uuid.uuid4().hex)
    spans: list[Span] = field(default_factory=list)

    def of_kind(self, kind: SpanKind) -> list[Span]:
        return [s for s in self.spans if s.kind == kind]

    def tool_calls(self) -> list[tuple[str, dict]]:
        return [(s.name, s.input) for s in self.of_kind("tool")]

    def total_tokens(self) -> int:
        return sum(s.attributes.get("tokens.total", 0) for s in self.of_kind("llm"))
```

The attribute names here mirror the **OpenTelemetry GenAI semantic conventions** (`gen_ai.request.model`, `gen_ai.usage.input_tokens`, and so on) closely enough that exporting to Langfuse, Arize Phoenix, LangSmith, or any OTel backend later is a mapping exercise, not a rewrite.

A context manager makes instrumentation nearly invisible:

``` python
from contextlib import contextmanager

class Tracer:
    def __init__(self):
        self.trace = Trace()
        self._stack: list[Span] = []

    @contextmanager
    def span(self, name: str, kind: SpanKind, input: Any = None, **attrs):
        s = Span(
            name=name, kind=kind, trace_id=self.trace.trace_id,
            parent_id=self._stack[-1].span_id if self._stack else None,
            input=input, attributes=dict(attrs),
        )
        self.trace.spans.append(s)
        self._stack.append(s)
        try:
            yield s
        except Exception as e:
            s.error = f"{type(e).__name__}: {e}"
            raise
        finally:
            s.end = time.time()
            self._stack.pop()
```

## The Model Seam

All model calls go through one client wrapper. It records the request and response as an `llm` span and can be swapped for a replay client in tests.

``` python
class ModelClient:
    def __init__(self, tracer: Tracer, model: str, llm_fn):
        self.tracer, self.model, self.llm_fn = tracer, model, llm_fn

    def complete(self, messages: list[dict], tools: list[dict]) -> dict:
        with self.tracer.span("chat", "llm", input=messages,
                              **{"gen_ai.request.model": self.model}) as s:
            resp = self.llm_fn(model=self.model, messages=messages, tools=tools)
            s.output = resp["message"]
            s.attributes["tokens.input"] = resp["usage"]["input_tokens"]
            s.attributes["tokens.output"] = resp["usage"]["output_tokens"]
            s.attributes["tokens.total"] = sum(resp["usage"].values())
            return resp["message"]
```

`llm_fn` is whatever calls your provider. Keeping it injected means the same agent runs against Claude, GPT, or a local model, and an eval can compare them with no code changes.

## The Tool Seam: Live, Mock, Replay

This is the most valuable seam. Evaluation needs to run tools in three modes:

- **Live** : real APIs, for end-to-end smoke tests in staging
- **Mock** : deterministic fakes backed by fixture data, for the bulk of offline evals
- **Replay** : return exactly what a recorded production trace returned, to reproduce a failure

``` python
from typing import Callable, Protocol

class ToolBackend(Protocol):
    def call(self, name: str, args: dict) -> Any: ...

class ToolExecutor:
    def __init__(self, tracer: Tracer, backend: ToolBackend, hooks=None):
        self.tracer, self.backend = tracer, backend
        self.hooks = hooks or []   # fault and adversarial injection, Lesson 9

    def run(self, name: str, args: dict) -> Any:
        with self.tracer.span(name, "tool", input=args) as s:
            result = self.backend.call(name, args)
            for hook in self.hooks:
                result = hook(name, args, result)
            s.output = result
            return result

class ReplayBackend:
    """Returns tool outputs recorded in a previous trace, in order."""
    def __init__(self, recorded: Trace):
        self._queue = [(s.name, s.input, s.output) for s in recorded.of_kind("tool")]

    def call(self, name, args):
        if not self._queue:
            raise RuntimeError(f"replay exhausted at {name}")
        exp_name, exp_args, output = self._queue.pop(0)
        if exp_name != name:
            raise RuntimeError(f"replay diverged: expected {exp_name}, got {name}")
        return output
```

Replay mode is how you turn a production bug report into a reproducible test case: take the trace, replay the tool outputs, and check whether a new prompt fixes the decision that went wrong.

## The State Seam

For Quill, the “world” is customers, invoices, and refunds. Put it behind an interface the mock backend reads and writes, and that evaluators can inspect.

```
@dataclass
class BillingState:
    customers: dict[str, dict]
    invoices: dict[str, dict]
    refunds: list[dict] = field(default_factory=list)
    escalations: list[dict] = field(default_factory=list)

    def snapshot(self) -> dict:
        import copy
        return copy.deepcopy(self.__dict__)

class MockBillingBackend:
    REFUND_LIMIT = 200.0

    def __init__(self, state: BillingState):
        self.state = state

    def call(self, name, args):
        s = self.state
        if name == "lookup_customer":
            return next((c for c in s.customers.values() if c["email"] == args["email"]),
                        {"error": "not_found"})
        if name == "get_invoices":
            return [i for i in s.invoices.values() if i["customer_id"] == args["customer_id"]]
        if name == "issue_refund":
            inv = s.invoices.get(args["invoice_id"])
            if inv is None:
                return {"error": "invoice_not_found"}
            if args["amount"] > self.REFUND_LIMIT:
                return {"error": "requires_approval"}
            s.refunds.append(dict(args))
            return {"status": "refunded", "refund_id": f"R{len(s.refunds)}"}
        if name == "escalate_to_human":
            s.escalations.append(dict(args))
            return {"status": "escalated"}
        if name == "search_kb":
            return [{"title": "Duplicate charges", "body": "Refund the duplicate within 30 days."}]
        return {"error": f"unknown_tool:{name}"}
```

Now an outcome grader can ask a precise question: *does `state.refunds` contain exactly one refund, for invoice INV-2291, for $49?* No parsing of prose required.

## Putting the Loop Together

``` python
import json

@dataclass
class RunResult:
    response: str
    trace: Trace
    final_state: dict
    stopped_reason: str     # "final_answer" | "max_steps" | "error"

class QuillAgent:
    def __init__(self, model: ModelClient, tools: ToolExecutor,
                 tracer: Tracer, system_prompt: str, max_steps: int = 10):
        self.model, self.tools, self.tracer = model, tools, tracer
        self.system_prompt, self.max_steps = system_prompt, max_steps

    def run(self, user_message: str, state: BillingState) -> RunResult:
        messages = [{"role": "system", "content": self.system_prompt},
                    {"role": "user", "content": user_message}]
        with self.tracer.span("quill", "agent", input=user_message) as root:
            for step in range(self.max_steps):
                msg = self.model.complete(messages, TOOL_SCHEMAS)
                messages.append(msg)
                if not msg.get("tool_calls"):
                    root.output = msg["content"]
                    return RunResult(msg["content"], self.tracer.trace,
                                     state.snapshot(), "final_answer")
                for call in msg["tool_calls"]:
                    result = self.tools.run(call["name"], call["arguments"])
                    messages.append({"role": "tool", "tool_call_id": call["id"],
                                     "content": json.dumps(result)})
            root.error = "max_steps"
            return RunResult("", self.tracer.trace, state.snapshot(), "max_steps")
```

Notice what an eval gets from a single `run()`:

- `response` : for response-quality graders
- `trace` : every LLM and tool span, for trajectory, cost, and latency graders
- `final_state` : for outcome assertions
- `stopped_reason` : for completion-rate metrics

## Architecture Decisions That Help (or Hurt) Evaluation

**Structured outputs over free text.** If the agent must end with a JSON object (`{"resolution": "refunded", "invoice_id": ...}`) alongside its message, you can grade the decision deterministically and the prose separately.

**Explicit stop reasons.** “Final answer,” “max steps,” “tool error,” and “guardrail blocked” are different failures. Collapse them into “it returned something” and your completion metric becomes meaningless.

**Config as data.** System prompt, model name, temperature, tool list, and max steps should be one versioned config object. Every eval result is then tagged with exactly what produced it, which is what makes comparing two versions possible (Lesson 12).

```
@dataclass(frozen=True)
class AgentConfig:
    model: str
    system_prompt_version: str
    temperature: float = 0.0
    max_steps: int = 10
    tools: tuple[str, ...] = ("lookup_customer", "get_invoices", "search_kb",
                              "issue_refund", "escalate_to_human")

    def fingerprint(self) -> str:
        import hashlib, json
        return hashlib.sha256(json.dumps(self.__dict__, sort_keys=True).encode()).hexdigest()[:12]
```

**Avoid hidden framework state.** Agent frameworks are fine, but make sure you can get the full message history and every tool call out of them. If a framework hides retries or silently truncates context, your traces will lie to you.

**Deterministic IDs and clocks in eval mode.** Inject the clock and ID generator. Otherwise “refund created at 14:03:22” makes snapshot comparisons flaky.

## Where Existing Tools Fit

| Seam | What to use | 
|---|---|
| Trace | OpenTelemetry with GenAI conventions; Langfuse, Arize Phoenix, LangSmith, Braintrust, Weights & Biases Weave as backends | 
| Model | Your own thin wrapper; LiteLLM if you need many providers | 
| Tool | Your own executor; MCP servers can sit behind it as a backend | 
| State | Your domain code, a test database, or containerized sandboxes for code and computer-use agents | 

The pattern is the same regardless: own the seams in your code, and let tools consume what flows through them.

## Key Takeaways

- Evaluation quality is capped by what the architecture exposes. Design for observability from day one.
- Build four explicit seams: **model** ,**tool** ,**state** , and**trace**
- A run should return the response, the full trace, the final state, and a stop reason
- Tool backends should support **live** ,**mock** , and**replay** modes
- Version the agent’s configuration as data, so every eval result knows exactly what it measured

## Exercise

Refactor an agent you’ve built (or implement Quill from the snippets above) so that:

1. Every LLM and tool call produces a span
2. Tools run against a mock backend backed by fixture state
3. `run()` returns a`RunResult` with trace, final state, and stop reason
4. You can take one recorded trace and replay it through `ReplayBackend`

Print a trace as an indented tree. If you can read the agent’s entire decision process from that tree, you’re ready for Lesson 3.
