Case Study: Pydantic AI Tool Context

Back to blog

June 14, 2026 · 4 min read

A measured Codex run fixing Pydantic AI tools that ignored per-run dependencies.

The fixture is a Pydantic AI support-routing agent.

Each run passes tenant-local data through Pydantic AI dependencies: tenant, region, ticket priorities, and escalation policies. The generated code created the agent with deps_type=SupportDeps

, but registered both tools with @agent.tool_plain

. Those tools cannot receive RunContext

, so they returned global fallback values.

Both runs used Codex GPT-5.5 against the same fixture. The prompt was:

Fix this Python fixture so `pytest` succeeds, preserving tenant-aware Pydantic AI routing tools.

The target package was pydantic-ai 2.0.0b7

.

Case study replay

Pydantic AI tenant routing tools #

model Codex GPT-5.5Fix this Python fixture so pytest succeeds, preserving tenant-aware Pydantic AI routing tools.

Without GitHits

• tokens

• 0

• time

• 0s / 189s

• Ready. Click "Watch Replay" to start.

• The Pydantic AI tools now use @agent.tool with RunContext[SupportDeps], so tenant, region, priorities, and escalation policies come from the active per-run dependencies.

With GitHits

• tokens

• 0

• time

• 0s / 99s

• Ready. Click "Watch Replay" to start.

• Used GitHits to confirm Pydantic AI 2.0.0b7's context-aware tool pattern, then changed the two tools from tool_plain to tool with RunContext[SupportDeps].

Result #

Run	Time	Tokens	Tools
With GitHits	99s	393,469	21
Without GitHits	189s	901,661	28

Both runs produced a passing patch. The GitHits run used 508,192 fewer processed tokens and finished 90 seconds sooner.

Failure #

The tools were registered with tool_plain

:

@agent.tool_plain
def lookup_ticket(ticket_id: int) -> str:
    return f"{DEFAULT_TENANT}:{ticket_id}:{DEFAULT_PRIORITY}"

@agent.tool_plain
def escalation_policy(ticket_id: int) -> str:
    return (
        f"{DEFAULT_TENANT}:{DEFAULT_PRIORITY}:"
        f"{DEFAULT_POLICY}:{DEFAULT_REGION}"
    )

tool_plain

is correct for tools that do not need the run context. These tools depend on SupportDeps

.

The tests checked data flow:

acme

andglobex

must route differently.lookup_ticket

andescalation_policy

must read the same per-run dependencies.- Unknown tickets should still fall back to normal

andstandard

, but the fallback must keep the active tenant and region.

Fix #

Use context-aware tools and read ctx.deps

:

from pydantic_ai import Agent, RunContext

@agent.tool
def lookup_ticket(ctx: RunContext[SupportDeps], ticket_id: int) -> str:
    priority = ctx.deps.priorities.get(ticket_id, DEFAULT_PRIORITY)
    return f"{ctx.deps.tenant}:{ticket_id}:{priority}"

@agent.tool
def escalation_policy(ctx: RunContext[SupportDeps], ticket_id: int) -> str:
    priority = ctx.deps.priorities.get(ticket_id, DEFAULT_PRIORITY)
    policy = ctx.deps.escalation_policies.get(priority, DEFAULT_POLICY)
    return f"{ctx.deps.tenant}:{priority}:{policy}:{ctx.deps.region}"

The patch depends on three package facts:

@agent.tool

is the decorator for tools that receiveRunContext

.RunContext[SupportDeps]

gives the tool access to the active dependency object.- Both tools need to read ctx.deps

; fixing only one still leaves inconsistent routing.

Trace #

The replay shows seven GitHits tool calls:

• Two search

calls for Pydantic AI docs aroundagent.tool

,RunContext

,deps

, andtool_plain

. - Three docs_read

calls on the current tools documentation. - One code_grep

call for thedef tool(

implementation. - One code_read

call on the package source whereAgent.tool

is defined.

Those calls gave the agent the package contract before editing: use @agent.tool

when a tool needs RunContext

, then access per-run dependencies through ctx.deps

.

The no-GitHits run had to reconstruct the same information from the local environment. It searched installed pydantic_ai

internals, found the package path, read exports, read agent source, read run-context source, patched, tested, cleaned local artifacts, reread the file, and tested again.

That local probing accounts for most of the 508k-token gap.

Evidence #

A passing fixture test proves the local behavior for the tested cases. The docs and source establish that the patch uses the intended Pydantic AI mechanism.

The GitHits trace had a short evidence chain:

• Docs showed the context-aware tool pattern for the current package.
• Source search found the Agent.tool

implementation surface. - Source reads confirmed that this was the right API boundary. pytest

verified tenant-specific behavior in the fixture.

The package has multiple valid tool decorators. The evidence points to the one that matches the data-flow requirement.

The final patch did not rewrite the routing model, change the tests, or move tenant data into prompts. It changed the decorator and dependency access in the two tools.

Accuracy Risk #

The incorrect alternatives are close to the correct patch:

• Keep tool_plain

and add globals. - Put tenant data into the prompt.

• Capture dependencies in a closure instead of using Pydantic AI’s run context.
• Fix lookup_ticket

but leaveescalation_policy

on defaults. - Change the tests to match global fallback behavior.

All of those preserve the original bug or create a more brittle fixture.

The GitHits run found the package mechanism in docs and source before editing. The no-GitHits run found the same mechanism by probing the installed package.

source & further reading

githits.com — original article Show HN: GitHits Public Beta 0.9 What It Actually Takes to Index Open Source Case Study: Persisted TanStack Query Mutations