We Are Putting Pydantic AI v2 in Three Places. None of Them Is the Orchestrator.

Pydantic AI v2 is being integrated into three specific layers of two production systems—a content-generation platform and a brand-measurement platform—but not as the orchestrator, according to a build status report from August 2026. The adapters are implementation-ready drafts, not merged, and no evaluations have been run. The design places Pydantic AI at the typed intelligence layer inside selected steps, while LangGraph remains the orchestrator for long-running workflows.

Build status — August 2026. Everything described here is designed against real seams in two production systems: a content-generation platform and a brand-measurement platform, both LangGraph-orchestrated, both running on FastAPI, ARQ, Redis and PostgreSQL. The workflows, classifiers and services are real code. The Pydantic AI adapters are implementation-ready drafts. They are not merged, and no evaluation has been run. There are no result tables below because there are no results yet. This is the interval between “this architecture makes sense” and “the evidence says it works,” and I think the interval is the part worth publishing. You do not wake up thinking you need another agent framework. You wake up thinking: why is this model returning almost-valid JSON again? Why does switching providers mean a new client path, a new parser, and a new family of failure modes? How do I test a stochastic classifier when one successful run is not evidence of anything? And how do I let an assistant trigger a product action without letting the model decide which tenant it belongs to? Meanwhile the discourse has turned this into a cage match. LangGraph in one corner, Pydantic AI in the other, and someone will eventually ask which one wins — because apparently software architecture is now marketed like professional wrestling. That was never the question. We already use LangGraph for long-running workflows: it owns state, conditional transitions, checkpointing, recovery, and the shape of multi-stage execution. Replacing it would solve a problem we do not have. The problem we do have sits one layer lower. Individual LLM operations still carry too much hand-written infrastructure. Prompts describe JSON. Code repairs JSON. Provider adapters disagree about structured output. Tests verify examples but cannot measure variance. Product services are typed and governed, but there is no safe model-facing surface over them. That is the layer we are giving to Pydantic AI. Not the conductor. The typed intelligence inside selected steps. The boundary came before the code The design is easier to defend if each layer has exactly one job. Three category errors this is meant to prevent: With the boundary fixed, three application points became obvious. They are ordered here as a risk ladder — leaf node first, governed action last — and that ordering is the actual plan, not a narrative device. Case one: replacing “return only valid JSON” with an output contract The first candidate was deliberately boring: the keyword-planning node in the content workflow. It runs after the outline is expanded into draft sections and before the drafter writes anything. It receives section indices, heading levels, keyword targets, priorities and heading-level constraints, and it decides which keyword belongs in which section. The prompt ended the way thousands of prompts end: Return ONLY valid JSON.{"assignments": {"0": "keyword a", "keyword b" ,"2": "keyword c" }} And the application did what applications do: php def parse llm assignments response: str - dict int, list str : text = response.strip text = re.sub r"^This is defensive code, and it works. It is also a parser for a convention described in prose. Every rule it enforces — the key is an integer, the value is a list of strings, the fences are noise — lives in the parser rather than in a contract the model was asked to satisfy. The replacement is an output model: python from pydantic import BaseModel, ConfigDict, Field, field validatorclass KeywordAssignmentPlan BaseModel : model config = ConfigDict extra="forbid" assignments: dict int, list str = Field default factory=dict @field validator "assignments" @classmethod def normalize cls, value: dict int, list str - dict int, list str : normalized: dict int, list str = {} for section index, keywords in value.items : cleaned = list dict.fromkeys keyword.strip for keyword in keywords if keyword and keyword.strip if cleaned: normalized section index = cleaned return normalized The adapter around it stays small, because it should: python from pydantic ai import Agentfrom pydantic ai.models.openrouter import OpenRouterModelfrom pydantic ai.providers.openrouter import OpenRouterProviderclass TypedKeywordPlanner: def init self, , model id: str, api key: str - None: self.agent = Agent OpenRouterModel model id, provider=OpenRouterProvider api key=api key , output type=KeywordAssignmentPlan, instructions=PLANNER SYSTEM PROMPT, retries={"output": 2}, async def assign self, user prompt: str - KeywordAssignmentPlan: result = await self.agent.run user prompt, model settings={"temperature": 0.2, "max tokens": 2200}, return result.output The second instance of this pattern is in the measurement platform, where a retrieval-query generator asks a model for a semantic query given a domain, a schema fragment and a document excerpt. There the interesting part is that the validator carries a retrieval constraint, not a transport constraint: class GeneratedRetrievalQuery BaseModel : query: str confidence: float = Field ge=0.0, le=1.0 @field validator "query" @classmethod def must fit retrieval contract cls, value: str - str: if not 10 <= len value.split <= 50: raise ValueError "query must contain between 10 and 50 words" return value.strip {“type”: “json object”} guarantees a shape at the transport boundary. It does not guarantee a query between ten and fifty words, or a confidence value inside its own range. Moving those into the output type moves the failure from “the parser found something unexpected” to “the run did not complete until the contract held, or the retry budget was exhausted.” The thing worth taking from this case is where the schema stops. The keyword plan model validates shape: an assignments object, integer-compatible keys, string lists, no extra top-level fields. It deliberately does not validate that the section index exists in the current outline, that the keyword comes from the supplied target set, that an H3-only keyword was not placed on an H2, or that the per-section cap holds. Those stay in deterministic Python, downstream, unchanged. A validated object is structurally trustworthy. It is not thereby correct, allowed, or safe to persist — and encoding domain rules into the output schema quietly moves business logic into a prompt-shaped artifact where it is much harder to test. One decision the two systems answered differently Both systems face the same rollout question: how do you compare the typed path against the legacy path without changing production behavior underneath yourself? The measurement platform runs shadow mode. Production returns the legacy result; the candidate runs against the same frozen input and both are persisted for comparison: php async def generate query with shadow , prompt: str, domain name: str - QueryResult: legacy = await legacy generator.generate prompt=prompt, domain name=domain name if settings.TYPED QUERY SHADOW ENABLED: candidate = await typed generator.generate prompt=prompt, domain name=domain name await shadow store.record legacy=legacy, candidate=candidate, document hash=document hash, model=model id, return legacy The content platform does not. It keeps a single call in the workflow and moves comparison into an offline A/B runner over frozen cases, behind a feature flag that defaults to legacy. The difference is not philosophical, it is arithmetic. A retrieval-query call is small, low-frequency and bounded; paying for it twice buys distribution data on live inputs cheaply. Keyword planning sits inside a bulk content pipeline where every generation job would silently double a model bill, and the inputs are reproducible enough that frozen fixtures lose almost nothing. Shadow mode is the better instrument when live input diversity is the thing you cannot reconstruct. An offline runner is the better instrument when you can reconstruct it and the traffic is expensive. Whichever you pick, the flag is a rollback boundary, not an architecture: KEYWORD PLANNER ENGINE: Literal "legacy", "pydantic ai" = "legacy" Rollback is that value returned to legacy. No migration, no graph change. Case two: making a stochastic classifier testable without making the score stochastic The second integration touches an answer-alignment classifier — the component that compares an external answer against a grounded reference and reports what is contradicted, omitted, or added. The invariant there predates any framework choice: LLMs classify, Python calculates. The model returns booleans and diagnostic evidence. Python computes the score. The formula does not come from the model, and the model is never invited to improvise a number that merely feels accurate. class AlignmentClassification BaseModel : core claims identified: list str | None = None is on topic: bool = True is factually correct: bool is complete: bool contradiction details: str | None = None omission details: str | None = None addition details: str | None = None addition count: int = Field default=0, ge=0 alignment classifier = Agent output type=AlignmentClassification, instructions= "Act only as a comparison engine. " "Return classifications and diagnostic evidence. " "Never assign a numeric alignment score." , The current implementation carries five provider-specific execution paths — native structured output on one vendor, a raw JSON fallback for the same vendor, and three others — each with its own request settings, response extraction, usage parsing and recovery behavior. That code exists for good reasons and it works. It also means one classification contract is surrounded by five different ways to fail. Making the classifier provider-neutral is worth something. But the reason this integration ranks second on the ladder rather than first is the evaluation stack, not the adapter. A unit test proves that a mocked response parses. It cannot tell you how often the real classifier changes its mind across five runs, which cases are unstable, or whether a provider switch introduces a contradiction bias. For a component whose worst client-facing failure is a false-positive contradiction — telling a customer a source contradicts them when it does not — “it parsed” is not a useful thing to know. So the dataset is code, and repetition is the measurement: python from pydantic evals import Case, Datasetalignment dataset = Dataset name="alignment-classification-shadow-v1", cases= Case name=fixture.case id, inputs=AlignmentEvalInput.model validate fixture.inputs , metadata={ "fixture sha256": fixture.sha256, "prompt version": "alignment v1", }, for fixture in frozen fixtures , Repeat is deliberate. One run is an anecdote.report = await alignment dataset.evaluate run case, repeat=5 The evaluators check behavioral contracts rather than prose equality. Was the output schema valid? Did the classifier preserve “unknown” versus “contradicted”? Did it flag a material contradiction consistently across runs? Did the deterministic score stay invariant given the same classification? How often did labels flip? What did each provider cost, in tokens and in p95 latency? The same idea appears in the content platform, where the planner evaluator scores against an acceptable set rather than an expected blob: class SemanticPlacementEvaluator Evaluator PlannerEvalInput, PlannerEvalOutput : def evaluate self, ctx: EvaluatorContext PlannerEvalInput, PlannerEvalOutput - float: allowed = { keyword.lower : set indices for keyword, indices in ctx.inputs.acceptable sections by keyword.items } total = accepted = 0 for section index, keywords in ctx.output.assignments.items : for keyword in keywords: total += 1 accepted += int section index in allowed.get keyword.lower , set return accepted / total if total else 0.0 Two keyword plans can both be defensible. Exact-match scoring would reward one arbitrary arrangement and punish another valid one, and you would spend a week tuning a prompt against an artifact of your fixture author’s taste. The limit is worth stating plainly, because it is the part framework adoption tends to blur: an evaluation framework does not create a gold set. One of our corpora is explicitly labelled as proxy structure rather than reviewed semantic gold, and its status stays blocked until operator annotation exists. Adopting Pydantic Evals does not change that. A nicer report object cannot promote proxy labels into truth. What the framework buys is a standard vocabulary for cases, repeats, evaluators and comparisons — so that experiments become comparable without a bespoke harness per component. Keeping gold, proxy and exploratory evidence distinguishable inside that vocabulary is still our job. Case three: the model can choose a tool. It cannot choose its authority. The third integration looks like the most agentic one, and it is where vocabulary gets dangerous. Pydantic AI v2 uses capability for a composable bundle that contributes tools, instructions, settings and lifecycle hooks to an agent. Our measurement platform’s gateway also uses capability — for a governed product contract carrying a risk class, a required access level, input and output schema versions, timeout, retry policy, quota behavior, audit requirements and a handler reference. These are not the same object, and conflating them would be convenient and wrong. One is an agent-runtime composition mechanism. The other is a policy artifact. The rule that follows is concrete: identity never arrives as a model-generated argument. Tenant, principal and workspace enter through typed dependencies resolved at an authenticated boundary. The model sees only the business parameters a user could legitimately choose. @dataclassclass BulkRunDeps: organization id: int user id: int run service: BulkRunServiceclass AssistantAnswer BaseModel : summary: str run id: int | None = None status: str | None = None blocked content ids: list int = Field default factory=list limitations: list str = Field default factory=list bulk run agent = Agent "openrouter:openai/gpt-5-mini", deps type=BulkRunDeps, output type= AssistantAnswer, DeferredToolRequests , instructions= "Use the bulk-run tools for the current organization only. " "Preview before proposing execution. Never claim a run started " "unless the start tool returned a run ID. Report blockers explicitly." , @bulk run agent.toolasync def preview generation ctx: RunContext BulkRunDeps , content ids: list int - list dict : """Preview assignments and blockers without executing anything.""" preview = await ctx.deps.run service.preview organization id=ctx.deps.organization id, content ids=content ids, return item.model dump mode="json" for item in preview @bulk run agent.tool requires approval=True async def start generation run ctx: RunContext BulkRunDeps , content ids: list int - dict: """Start an approved run for previewed content.""" result = await ctx.deps.run service.create and enqueue run organization id=ctx.deps.organization id, content ids=content ids, user id=ctx.deps.user id, return result.model dump mode="json" Two details in that snippet are load-bearing. First, requires approval=True does not block inline and wait. The run ends , returning a DeferredToolRequests object — which is why DeferredToolRequests has to be declared in output type. The caller renders each pending call to a human, collects decisions, and resumes with DeferredToolResults plus the previous message history. It is a two-run protocol, and treating it as a synchronous pause is the fastest way to ship an assistant that appears to hang. Second, and more important: approval is not authorization. They are different guarantees and most agent write-ups collapse them. Authorization already happened, before the agent existed. The API boundary authenticated the user and resolved organization membership before constructing the dependencies. The service filters content by organization. The assignment service loads organization-scoped records. Locks and queue state determine whether work can start at all. On the measurement side, the equivalent path checks the capability grant, validates unknown fields against the registered input model, claims or replays an idempotency key, consumes quota, writes an append-only invocation record, applies bounded timeout and retry, validates the output contract and redacts the operator-facing result. Approval adds one thing on top: a human sees the exact tool name and the validated arguments before a side effect occurs. Remove approval and the system is still safe. Remove authorization and approval is theater — a confirmation dialog over an action nobody checked you were allowed to take. There is a third guarantee hiding in the output type. AssistantAnswer requires a run id and a status alongside the summary, and the instructions tie the success claim to a tool result. That does not make the model honest. It does make the dishonest version structurally awkward: a confident paragraph saying “your run has started” has to survive a contract that wants the ID of the run it claims started. What we are deliberately not giving it Every framework ships a list of impressive features. Mature systems need the second list — the responsibilities the framework is not allowed to absorb. It does not calculate the alignment score. Python does. It does not decide whether a claim is approved, publishable or safe for release. Existing gates do. It does not resolve tenant access, quota, delegation or workspace scope. The gateway does. It does not replace ARQ for durable jobs or the PostgreSQL-backed checkpointer for existing workflows. We are not migrating LangGraph to Pydantic Graph. It is not being piloted on our hardest surface. The conversational analytics agent has Redis-backed conversation checkpointing, streaming, retrieval tools and a long history of tool-loop edge cases. That is not where you learn a new runtime boundary. And it does not turn a successful fixture into a benchmark. What has to be in the next article The next one replaces intent with evidence, or it should not be published. For the typed planner: fixture count and hashes, schema-validity and retry distribution, high-priority coverage, semantic placement against acceptable sets, cross-run Jaccard stability, fallback rate, p50/p95 latency, tokens, cost, and the cases where the legacy and typed paths disagreed. For the classifier: multi-run label stability, disagreement by case type, provider variance, deterministic-score invariance, and an error taxonomy rather than an average. For the assistant: tool-selection accuracy, cross-organization isolation tests, approval/denial/modified-argument flows, duplicate-start and lock-conflict behavior, and evidence that it cannot narrate a failed enqueue as a success. Until those artifacts exist, the honest claim is architectural: these are the seams where a typed agent layer adds leverage without absorbing responsibilities the platform already handles. That is a smaller claim than most framework posts make, and it is the one I can currently defend. Building in public that skips this interval is not building in public. It is publishing the victory speech and deleting the lab notebook. One workflow, typed intelligence, governed action Strip the framework names out and the design is small. LangGraph owns where the process goes. Pydantic AI owns how one model-facing step reaches a typed result or selects a narrow tool. Pydantic Evals owns repeatable experiments. Product services own what actually happens. Python owns the rules that must reproduce exactly. That division will not satisfy anyone looking for one abstraction to swallow the stack. It is not supposed to. “One framework to rule them all” is not an architecture requirement; it is a conference slide. So here is the boundary I would like other builders to attack. Where does probabilistic judgment stop in your system, and what takes authority from there? When does a composable agent capability become a governed product capability? Which validation failure belongs inside the model loop, and which one has to terminate in deterministic code? We have drawn the line around typed generation, stochastic classification and tool selection. Show me the tool in your agent system that cannot execute until both policy and a human agree. Then come back for the evals, because that is where the argument becomes evidence. References TES — The Entity Swiss Knife · Max Geraci We Are Putting Pydantic AI v2 in Three Places. None of Them Is the Orchestrator. https://pub.towardsai.net/we-are-putting-pydantic-ai-v2-in-three-places-none-of-them-is-the-orchestrator-811a6fc04208 was originally published in Towards AI https://pub.towardsai.net on Medium, where people are continuing the conversation by highlighting and responding to this story.