{"slug": "we-are-putting-pydantic-ai-v2-in-three-places-none-of-them-is-the-orchestrator", "title": "We Are Putting Pydantic AI v2 in Three Places. None of Them Is the Orchestrator.", "summary": "Pydantic AI v2 is being integrated into three specific layers of two production systems—a content-generation platform and a brand-measurement platform—but not as the orchestrator, according to a build status report from August 2026. The adapters are implementation-ready drafts, not merged, and no evaluations have been run. The design places Pydantic AI at the typed intelligence layer inside selected steps, while LangGraph remains the orchestrator for long-running workflows.", "body_md": "**Build status — August 2026.** Everything described here is designed against real seams in two production systems: a content-generation platform and a brand-measurement platform, both LangGraph-orchestrated, both running on FastAPI, ARQ, Redis and PostgreSQL. The workflows, classifiers and services are real code. The Pydantic AI adapters are implementation-ready drafts. **They are not merged, and no evaluation has been run.** There are no result tables below because there are no results yet. This is the interval between “this architecture makes sense” and “the evidence says it works,” and I think the interval is the part worth publishing.\n\nYou do not wake up thinking you need another agent framework.\n\nYou wake up thinking: why is this model returning almost-valid JSON again? Why does switching providers mean a new client path, a new parser, and a new family of failure modes? How do I test a stochastic classifier when one successful run is not evidence of anything? And how do I let an assistant trigger a product action without letting the model decide which tenant it belongs to?\n\nMeanwhile the discourse has turned this into a cage match. LangGraph in one corner, Pydantic AI in the other, and someone will eventually ask which one wins — because apparently software architecture is now marketed like professional wrestling.\n\nThat was never the question. We already use LangGraph for long-running workflows: it owns state, conditional transitions, checkpointing, recovery, and the shape of multi-stage execution. Replacing it would solve a problem we do not have.\n\nThe problem we do have sits one layer lower. Individual LLM operations still carry too much hand-written infrastructure. Prompts describe JSON. Code repairs JSON. Provider adapters disagree about structured output. Tests verify examples but cannot measure variance. Product services are typed and governed, but there is no safe model-facing surface over them.\n\nThat is the layer we are giving to Pydantic AI. Not the conductor. The typed intelligence inside selected steps.\n\n**The boundary came before the code**\n\nThe design is easier to defend if each layer has exactly one job.\n\nThree category errors this is meant to prevent:\n\nWith the boundary fixed, three application points became obvious. They are ordered here as a risk ladder — leaf node first, governed action last — and that ordering is the actual plan, not a narrative device.\n\n**Case one: replacing “return only valid JSON” with an output contract**\n\nThe first candidate was deliberately boring: the keyword-planning node in the content workflow. It runs after the outline is expanded into draft sections and before the drafter writes anything. It receives section indices, heading levels, keyword targets, priorities and heading-level constraints, and it decides which keyword belongs in which section.\n\nThe prompt ended the way thousands of prompts end:\n\n```\nReturn ONLY valid JSON.{\"assignments\": {\"0\": [\"keyword a\", \"keyword b\"],\"2\": [\"keyword c\"]}}\n```\n\nAnd the application did what applications do:\n\n``` php\ndef _parse_llm_assignments(response: str) -> dict[int, list[str]]:  text = response.strip()  text = re.sub(r\"^This is defensive code, and it works. It is also a parser for a convention described in prose. Every rule it enforces — the key is an integer, the value is a list of strings, the fences are noise — lives in the parser rather than in a contract the model was asked to satisfy.\n\nThe replacement is an output model:\n\n``` python\nfrom pydantic import BaseModel, ConfigDict, Field, field_validatorclass KeywordAssignmentPlan(BaseModel):    model_config = ConfigDict(extra=\"forbid\")    assignments: dict[int, list[str]] = Field(default_factory=dict)    @field_validator(\"assignments\")    @classmethod    def normalize(cls, value: dict[int, list[str]]) -> dict[int, list[str]]:        normalized: dict[int, list[str]] = {}        for section_index, keywords in value.items():            cleaned = list(dict.fromkeys(                keyword.strip() for keyword in keywords if keyword and keyword.strip()            ))            if cleaned:                normalized[section_index] = cleaned        return normalized\n```\n\nThe adapter around it stays small, because it should:\n\n``` python\nfrom pydantic_ai import Agentfrom pydantic_ai.models.openrouter import OpenRouterModelfrom pydantic_ai.providers.openrouter import OpenRouterProviderclass TypedKeywordPlanner:    def __init__(self, *, model_id: str, api_key: str) -> None:        self.agent = Agent(            OpenRouterModel(model_id, provider=OpenRouterProvider(api_key=api_key)),            output_type=KeywordAssignmentPlan,            instructions=PLANNER_SYSTEM_PROMPT,            retries={\"output\": 2},        )    async def assign(self, user_prompt: str) -> KeywordAssignmentPlan:        result = await self.agent.run(            user_prompt,            model_settings={\"temperature\": 0.2, \"max_tokens\": 2200},        )        return result.output\n```\n\nThe second instance of this pattern is in the measurement platform, where a retrieval-query generator asks a model for a semantic query given a domain, a schema fragment and a document excerpt. There the interesting part is that the validator carries a *retrieval* constraint, not a transport constraint:\n\n```\nclass GeneratedRetrievalQuery(BaseModel):    query: str    confidence: float = Field(ge=0.0, le=1.0)    @field_validator(\"query\")    @classmethod    def must_fit_retrieval_contract(cls, value: str) -> str:        if not 10 <= len(value.split()) <= 50:            raise ValueError(\"query must contain between 10 and 50 words\")        return value.strip()\n```\n\n{“type”: “json_object”} guarantees a shape at the transport boundary. It does not guarantee a query between ten and fifty words, or a confidence value inside its own range. Moving those into the output type moves the failure from “the parser found something unexpected” to “the run did not complete until the contract held, or the retry budget was exhausted.”\n\n**The thing worth taking from this case is where the schema stops.** The keyword plan model validates shape: an assignments object, integer-compatible keys, string lists, no extra top-level fields. It deliberately does not validate that the section index exists in the current outline, that the keyword comes from the supplied target set, that an H3-only keyword was not placed on an H2, or that the per-section cap holds. Those stay in deterministic Python, downstream, unchanged. A validated object is structurally trustworthy. It is not thereby correct, allowed, or safe to persist — and encoding domain rules into the output schema quietly moves business logic into a prompt-shaped artifact where it is much harder to test.\n\n**One decision the two systems answered differently**\n\nBoth systems face the same rollout question: how do you compare the typed path against the legacy path without changing production behavior underneath yourself?\n\nThe measurement platform runs shadow mode. Production returns the legacy result; the candidate runs against the same frozen input and both are persisted for comparison:\n\n``` php\nasync def generate_query_with_shadow(*, prompt: str, domain_name: str) -> QueryResult:    legacy = await legacy_generator.generate(prompt=prompt, domain_name=domain_name)    if settings.TYPED_QUERY_SHADOW_ENABLED:        candidate = await typed_generator.generate(prompt=prompt, domain_name=domain_name)        await shadow_store.record(            legacy=legacy, candidate=candidate,            document_hash=document_hash, model=model_id,        )    return legacy\n```\n\nThe content platform does not. It keeps a single call in the workflow and moves comparison into an offline A/B runner over frozen cases, behind a feature flag that defaults to legacy.\n\nThe difference is not philosophical, it is arithmetic. A retrieval-query call is small, low-frequency and bounded; paying for it twice buys distribution data on live inputs cheaply. Keyword planning sits inside a bulk content pipeline where every generation job would silently double a model bill, and the inputs are reproducible enough that frozen fixtures lose almost nothing. Shadow mode is the better instrument when live input diversity is the thing you cannot reconstruct. An offline runner is the better instrument when you can reconstruct it and the traffic is expensive.\n\nWhichever you pick, the flag is a rollback boundary, not an architecture:\n\n```\nKEYWORD_PLANNER_ENGINE: Literal[\"legacy\", \"pydantic_ai\"] = \"legacy\"\n```\n\nRollback is that value returned to legacy. No migration, no graph change.\n\n**Case two: making a stochastic classifier testable without making the score stochastic**\n\nThe second integration touches an answer-alignment classifier — the component that compares an external answer against a grounded reference and reports what is contradicted, omitted, or added.\n\nThe invariant there predates any framework choice: **LLMs classify, Python calculates.** The model returns booleans and diagnostic evidence. Python computes the score. The formula does not come from the model, and the model is never invited to improvise a number that merely feels accurate.\n\n```\nclass AlignmentClassification(BaseModel):    core_claims_identified: list[str] | None = None    is_on_topic: bool = True    is_factually_correct: bool    is_complete: bool    contradiction_details: str | None = None    omission_details: str | None = None    addition_details: str | None = None    addition_count: int = Field(default=0, ge=0)alignment_classifier = Agent(    output_type=AlignmentClassification,    instructions=(        \"Act only as a comparison engine. \"        \"Return classifications and diagnostic evidence. \"        \"Never assign a numeric alignment score.\"    ),)\n```\n\nThe current implementation carries five provider-specific execution paths — native structured output on one vendor, a raw JSON fallback for the same vendor, and three others — each with its own request settings, response extraction, usage parsing and recovery behavior. That code exists for good reasons and it works. It also means one classification contract is surrounded by five different ways to fail.\n\nMaking the classifier provider-neutral is worth something. But the reason this integration ranks second on the ladder rather than first is the evaluation stack, not the adapter.\n\nA unit test proves that a mocked response parses. It cannot tell you how often the real classifier changes its mind across five runs, which cases are unstable, or whether a provider switch introduces a contradiction bias. For a component whose worst client-facing failure is a false-positive contradiction — telling a customer a source contradicts them when it does not — “it parsed” is not a useful thing to know.\n\nSo the dataset is code, and repetition is the measurement:\n\n``` python\nfrom pydantic_evals import Case, Datasetalignment_dataset = Dataset(    name=\"alignment-classification-shadow-v1\",    cases=[        Case(            name=fixture.case_id,            inputs=AlignmentEvalInput.model_validate(fixture.inputs),            metadata={                \"fixture_sha256\": fixture.sha256,                \"prompt_version\": \"alignment_v1\",            },        )        for fixture in frozen_fixtures    ],)# Repeat is deliberate. One run is an anecdote.report = await alignment_dataset.evaluate(run_case, repeat=5)\n```\n\nThe evaluators check behavioral contracts rather than prose equality. Was the output schema valid? Did the classifier preserve “unknown” versus “contradicted”? Did it flag a material contradiction consistently across runs? Did the deterministic score stay invariant given the same classification? How often did labels flip? What did each provider cost, in tokens and in p95 latency?\n\nThe same idea appears in the content platform, where the planner evaluator scores against an *acceptable set* rather than an expected blob:\n\n```\nclass SemanticPlacementEvaluator(Evaluator[PlannerEvalInput, PlannerEvalOutput]):    def evaluate(self, ctx: EvaluatorContext[PlannerEvalInput, PlannerEvalOutput]) -> float:        allowed = {            keyword.lower(): set(indices)            for keyword, indices in ctx.inputs.acceptable_sections_by_keyword.items()        }        total = accepted = 0        for section_index, keywords in ctx.output.assignments.items():            for keyword in keywords:                total += 1                accepted += int(section_index in allowed.get(keyword.lower(), set()))        return accepted / total if total else 0.0\n```\n\nTwo keyword plans can both be defensible. Exact-match scoring would reward one arbitrary arrangement and punish another valid one, and you would spend a week tuning a prompt against an artifact of your fixture author’s taste.\n\n**The limit is worth stating plainly, because it is the part framework adoption tends to blur: an evaluation framework does not create a gold set.** One of our corpora is explicitly labelled as proxy structure rather than reviewed semantic gold, and its status stays blocked until operator annotation exists. Adopting Pydantic Evals does not change that. A nicer report object cannot promote proxy labels into truth. What the framework buys is a standard vocabulary for cases, repeats, evaluators and comparisons — so that experiments become comparable without a bespoke harness per component. Keeping gold, proxy and exploratory evidence distinguishable inside that vocabulary is still our job.\n\n**Case three: the model can choose a tool. It cannot choose its authority.**\n\nThe third integration looks like the most agentic one, and it is where vocabulary gets dangerous.\n\nPydantic AI v2 uses *capability* for a composable bundle that contributes tools, instructions, settings and lifecycle hooks to an agent. Our measurement platform’s gateway also uses *capability* — for a governed product contract carrying a risk class, a required access level, input and output schema versions, timeout, retry policy, quota behavior, audit requirements and a handler reference.\n\nThese are not the same object, and conflating them would be convenient and wrong. One is an agent-runtime composition mechanism. The other is a policy artifact.\n\nThe rule that follows is concrete: identity never arrives as a model-generated argument. Tenant, principal and workspace enter through typed dependencies resolved at an authenticated boundary. The model sees only the business parameters a user could legitimately choose.\n\n```\n@dataclassclass BulkRunDeps:    organization_id: int    user_id: int    run_service: BulkRunServiceclass AssistantAnswer(BaseModel):    summary: str    run_id: int | None = None    status: str | None = None    blocked_content_ids: list[int] = Field(default_factory=list)    limitations: list[str] = Field(default_factory=list)bulk_run_agent = Agent(    \"openrouter:openai/gpt-5-mini\",    deps_type=BulkRunDeps,    output_type=[AssistantAnswer, DeferredToolRequests],    instructions=(        \"Use the bulk-run tools for the current organization only. \"        \"Preview before proposing execution. Never claim a run started \"        \"unless the start tool returned a run ID. Report blockers explicitly.\"    ),)@bulk_run_agent.toolasync def preview_generation(ctx: RunContext[BulkRunDeps], content_ids: list[int]) -> list[dict]:    \"\"\"Preview assignments and blockers without executing anything.\"\"\"    preview = await ctx.deps.run_service.preview(        organization_id=ctx.deps.organization_id,        content_ids=content_ids,    )    return [item.model_dump(mode=\"json\") for item in preview]@bulk_run_agent.tool(requires_approval=True)async def start_generation_run(ctx: RunContext[BulkRunDeps], content_ids: list[int]) -> dict:    \"\"\"Start an approved run for previewed content.\"\"\"    result = await ctx.deps.run_service.create_and_enqueue_run(        organization_id=ctx.deps.organization_id,        content_ids=content_ids,        user_id=ctx.deps.user_id,    )    return result.model_dump(mode=\"json\")\n```\n\nTwo details in that snippet are load-bearing.\n\nFirst, requires_approval=True does not block inline and wait. The run *ends*, returning a DeferredToolRequests object — which is why DeferredToolRequests has to be declared in output_type. The caller renders each pending call to a human, collects decisions, and resumes with DeferredToolResults plus the previous message_history. It is a two-run protocol, and treating it as a synchronous pause is the fastest way to ship an assistant that appears to hang.\n\nSecond, and more important: **approval is not authorization.** They are different guarantees and most agent write-ups collapse them.\n\nAuthorization already happened, before the agent existed. The API boundary authenticated the user and resolved organization membership before constructing the dependencies. The service filters content by organization. The assignment service loads organization-scoped records. Locks and queue state determine whether work can start at all. On the measurement side, the equivalent path checks the capability grant, validates unknown fields against the registered input model, claims or replays an idempotency key, consumes quota, writes an append-only invocation record, applies bounded timeout and retry, validates the output contract and redacts the operator-facing result.\n\nApproval adds one thing on top: a human sees the exact tool name and the validated arguments before a side effect occurs. Remove approval and the system is still safe. Remove authorization and approval is theater — a confirmation dialog over an action nobody checked you were allowed to take.\n\nThere is a third guarantee hiding in the output type. AssistantAnswer requires a run_id and a status alongside the summary, and the instructions tie the success claim to a tool result. That does not make the model honest. It does make the dishonest version structurally awkward: a confident paragraph saying “your run has started” has to survive a contract that wants the ID of the run it claims started.\n\n**What we are deliberately not giving it**\n\nEvery framework ships a list of impressive features. Mature systems need the second list — the responsibilities the framework is not allowed to absorb.\n\nIt does not calculate the alignment score. Python does.\n\nIt does not decide whether a claim is approved, publishable or safe for release. Existing gates do.\n\nIt does not resolve tenant access, quota, delegation or workspace scope. The gateway does.\n\nIt does not replace ARQ for durable jobs or the PostgreSQL-backed checkpointer for existing workflows. We are not migrating LangGraph to Pydantic Graph.\n\nIt is not being piloted on our hardest surface. The conversational analytics agent has Redis-backed conversation checkpointing, streaming, retrieval tools and a long history of tool-loop edge cases. That is not where you learn a new runtime boundary.\n\nAnd it does not turn a successful fixture into a benchmark.\n\n**What has to be in the next article**\n\nThe next one replaces intent with evidence, or it should not be published.\n\nFor the typed planner: fixture count and hashes, schema-validity and retry distribution, high-priority coverage, semantic placement against acceptable sets, cross-run Jaccard stability, fallback rate, p50/p95 latency, tokens, cost, and the cases where the legacy and typed paths disagreed.\n\nFor the classifier: multi-run label stability, disagreement by case type, provider variance, deterministic-score invariance, and an error taxonomy rather than an average.\n\nFor the assistant: tool-selection accuracy, cross-organization isolation tests, approval/denial/modified-argument flows, duplicate-start and lock-conflict behavior, and evidence that it cannot narrate a failed enqueue as a success.\n\nUntil those artifacts exist, the honest claim is architectural: these are the seams where a typed agent layer adds leverage without absorbing responsibilities the platform already handles. That is a smaller claim than most framework posts make, and it is the one I can currently defend.\n\nBuilding in public that skips this interval is not building in public. It is publishing the victory speech and deleting the lab notebook.\n\n**One workflow, typed intelligence, governed action**\n\nStrip the framework names out and the design is small.\n\nLangGraph owns where the process goes. Pydantic AI owns how one model-facing step reaches a typed result or selects a narrow tool. Pydantic Evals owns repeatable experiments. Product services own what actually happens. Python owns the rules that must reproduce exactly.\n\nThat division will not satisfy anyone looking for one abstraction to swallow the stack. It is not supposed to. “One framework to rule them all” is not an architecture requirement; it is a conference slide.\n\nSo here is the boundary I would like other builders to attack. Where does probabilistic judgment stop in your system, and what takes authority from there? When does a composable agent capability become a governed product capability? Which validation failure belongs inside the model loop, and which one has to terminate in deterministic code?\n\nWe have drawn the line around typed generation, stochastic classification and tool selection. Show me the tool in your agent system that cannot execute until both policy and a human agree.\n\nThen come back for the evals, because that is where the argument becomes evidence.\n\n**References**\n\n*TES — The Entity Swiss Knife · Max Geraci*\n\n[We Are Putting Pydantic AI v2 in Three Places. None of Them Is the Orchestrator.](https://pub.towardsai.net/we-are-putting-pydantic-ai-v2-in-three-places-none-of-them-is-the-orchestrator-811a6fc04208) was originally published in [Towards AI](https://pub.towardsai.net) on Medium, where people are continuing the conversation by highlighting and responding to this story.", "url": "https://wpnews.pro/news/we-are-putting-pydantic-ai-v2-in-three-places-none-of-them-is-the-orchestrator", "canonical_source": "https://pub.towardsai.net/we-are-putting-pydantic-ai-v2-in-three-places-none-of-them-is-the-orchestrator-811a6fc04208?source=rss----98111c9905da---4", "published_at": "2026-08-02 12:01:01+00:00", "updated_at": "2026-08-02 12:22:00.564100+00:00", "lang": "en", "topics": ["artificial-intelligence", "ai-tools", "developer-tools"], "entities": ["Pydantic AI", "LangGraph", "FastAPI", "ARQ", "Redis", "PostgreSQL"], "alternates": {"html": "https://wpnews.pro/news/we-are-putting-pydantic-ai-v2-in-three-places-none-of-them-is-the-orchestrator", "markdown": "https://wpnews.pro/news/we-are-putting-pydantic-ai-v2-in-three-places-none-of-them-is-the-orchestrator.md", "text": "https://wpnews.pro/news/we-are-putting-pydantic-ai-v2-in-three-places-none-of-them-is-the-orchestrator.txt", "jsonld": "https://wpnews.pro/news/we-are-putting-pydantic-ai-v2-in-three-places-none-of-them-is-the-orchestrator.jsonld"}}