How To Evaluate An AI Agent

Pydantic AI introduced Pydantic Evals, an evaluation library designed to test AI agents by combining deterministic structural checks with LLM-based judgment for subjective qualities. The library includes evaluators such as IsInstance for type validation and LLMJudge for rubric-based scoring, addressing the challenges of non-deterministic outputs and subjective criteria in agent testing.

In classical test-driven development, we deal with deterministic outcomes. We write assertions against values and types we already know in advance — assert result == expected , and we move on. AI agents don't play by those rules. Because the underlying LLM is non-deterministic, giving it the same input twice can produce two differently-worded outputs. That alone breaks the classical assertion model. But there's a second problem, beyond just consistency. Say we have a customer support agent, and we want it to be helpful, empathetic, professional, and friendly when it talks to customers. Even if the output were consistent, how do we test for qualities like that? There's no fixed value or type to assert against — "empathetic" isn't a type or an exact string, it's a subjective judgment call. So we're dealing with two distinct problems: unpredictable output, and qualities that are inherently subjective. Testing an AI agent means solving for both. Pydantic Evals https://pydantic.dev/docs/ai/evals/evals/ is Pydantic AI's recommended way to test the output of your AI agents. It's an evaluation library that gives you a set of evaluators purpose-built for exactly this kind of testing — some for the deterministic parts of your agent's behavior, and some for the parts that require judgment. Not everything about an AI agent is unpredictable. If your agent is set up with a structured output type, you can — and should — still test that the shape of the response is correct, even if the content varies. For example, say we have an agent that returns a structured output type: python from pydantic ai import Agent from dataclasses import dataclass Response Model @dataclass class Response: message: str sentiment: str Agent Setup support agent = Agent model="openai/gpt-5.5", name="customer support agent", instructions=""" You are a customer support agent for an online store. """, output type=Response We can define a test case that checks the output is an instance of that Response type, using the IsInstance evaluator: python from pydantic evals import Case from pydantic evals.evaluators import IsInstance test case = Case name="check agent response", inputs="where is my order?", evaluators= IsInstance type name='Response' This is a classic assertion in spirit — we know exactly what type we expect back, so we test for it directly. No judgment call required. Structural checks like IsInstance don't help us with the harder problem: was the response actually good ? Was it empathetic? Professional? Did it address the customer's question? This is where LLMJudge comes in. It uses an LLM as a judge to evaluate your agent's output against a set of criteria you define — scoring qualitative aspects of the response that a type check could never catch. python from pydantic evals.evaluators import LLMJudge quality case = Case name="check response quality", inputs="where is my order?", evaluators= LLMJudge rubric="Response should be empathetic, professional, and directly address the customer's question about their order status." Under the hood, LLMJudge sends your agent's output and your rubric to a judge model, which returns a score and often a reason reflecting how well the response meets your criteria. Rather than asserting on an exact value, you're asserting on a standard . Together, IsInstance and LLMJudge cover both problems from earlier: structural correctness for the deterministic parts, and rubric-based scoring for everything else. I go deeper into LLMJudge — including setting up rubrics, reading scores, and combining multiple evaluators in a single test suite — in this video, part of the Master Pydantic AI series on my YouTube channel https://youtube.com/@joxiahdev . If you find it useful, I'd really appreciate a like and a subscribe on the channel.