{"slug": "why-i-refused-to-expose-pydantic-ai-to-my-frontend-research-notes", "title": "Why I Refused to Expose Pydantic AI to My Frontend — Research Notes", "summary": "A developer building ScriptorDB refused to expose Pydantic AI's internal event stream to the frontend, instead creating an application-level event translation layer that converts raw events into six stable frontend events pushed via SSE. This architectural decision decouples the frontend and persistence layer from the LLM framework's internal data structures, enabling real-time execution visibility for users while maintaining flexibility to swap protocols or frameworks.", "body_md": "In my previous article, I explained why Pydantic AI accounts for only 73 lines of ScriptorDB’s codebase. The remaining engineering effort lies in everything surrounding the framework.\n\nOne of those pieces is streaming.\n\nAlthough agent.run() can execute an Agent in a single call, building a production frontend requires far more than a final return value. That led us to build an application-level event translation layer instead of exposing Pydantic AI’s internal event stream directly.\n\nThis article explains why.\n\n[View the previous article in medium](https://medium.com/towards-artificial-intelligence/pydantic-ai-is-only-73-lines-of-my-codebase-the-other-90-is-architecture-ba8e52197b07)\n\nWe built an application-level event translation layer for ScriptorDB: the backend listens to Pydantic AI’s raw event stream, translates it into six stable frontend events ( run_start, text_delta, tool_call, tool_result, trace, run_end), and then pushes them via SSE (Server-Sent Events).\n\nThe core of this decision isn’t about showing off technically; it’s about adhering to an architectural principle: **never expose the LLM framework’s internal data structures to the frontend or the persistence layer.** As a result, the frontend only needs a dead-simple reducer to manage state, the persistence format is decoupled from the framework version, and the protocol layer (SSE) can be swapped out at any time without affecting inference or state logic.\n\nComparison of a static CLI output versus a streaming ScriptorDB web interface showing real-time tool call progress.\n\nPydantic AI can execute an agent with a single call to **agent.run()**. That works perfectly for a CLI, where the only thing you care about is **the final answer**.\n\n**A production web application is different**. Users don’t just wait for a result-they expect to see **the execution unfold**. They want to know which tool is running, whether it succeeds or fails, and why a run stops unexpectedly.\n\nIn ScriptorDB’s earliest CLI prototype, we simply called agent.run(), waited for an AgentRunResult, and printed result.data. There was no frontend, no sessions, and no streaming state. One line of code was enough.\n\nThat assumption broke down as soon as we introduced a React frontend. A request like “Look up the top five products by revenue last quarter and draw a trend chart” might trigger multiple tool calls, each taking several seconds. If a database query fails halfway through, the user shouldn’t stare at a loading spinner until a generic “Run Error” finally appears. The interface should reveal **what is happening while the Agent is still running**.\n\n**In short, a CLI only needs the result. A production product needs the process**.\n\nOnce we accepted that a frontend needs execution visibility, the most obvious solution seemed straightforward: serialize result.new_messages() and send it directly to the frontend:\n\nWait foragent.run() to finish, serializenew_messages into JSON, return it to the frontend via an API, and let the frontend parse out what tools were called and what text was output.\n\nWe didn’t get far before abandoning this approach. The reasons were highly practical:\n\n**First, ****new_messages is an internal framework structure.** When you call result.new_messages(), you receive a list of ModelMessage objects. Ignoring some internal fields, a simplified serialized version looks roughly like this:\n\n```\n[  {\"kind\": \"request\", \"parts\": [{\"content\": \"Look up last quarter's revenue...\"}]},  {\"kind\": \"response\", \"parts\": [{\"tool_name\": \"query_database\", \"args\": {\"sql\": \"SELECT ...\"}}]},  {\"kind\": \"request\", \"parts\": [{\"tool_return\": \"...\", \"tool_name\": \"query_database\"}]},  {\"kind\": \"response\", \"parts\": [{\"content\": \"Based on the query results...\"}]}]\n```\n\nThis is Pydantic AI’s own messaging protocol. If the frontend receives it, it has to understand concepts like kind, parts, and tool_return before it can figure out \"what tool was called and whether it succeeded or failed.\" Furthermore, field naming, serialization methods, and even type annotations might change as the framework upgrades. If we pass this directly to the frontend and persist it to disk, we are essentially surrendering our \"format sovereignty.\"\n\n**Second, the frontend doesn’t need to know the LLM message format.** The frontend only cares about: what the tool_name is, whether the status is running or success, how long it took to execute, what the output is, and what the error_code is. The ToolReturnPart only contains the serialized tool result; it lacks unified execution metadata.\n\n**Third, the routing layer would become a hodgepodge.** If we let the FastAPI route call agent.run() directly, the routing code would simultaneously take on inference invocation, protocol conversion, persistence logic, and error handling. That is not a router's job.\n\nSo we made a decision: **under no circumstances will we expose Pydantic AI’s internal data structures directly to the frontend or the persistence layer.** We needed our own event contract.\n\nBefore writing any code, we asked ourselves: what exactly does the frontend need to know?\n\nThe answer was simpler than we imagined:\n\nThat’s it. The frontend doesn’t need to know framework concepts like ModelRequest, ToolReturnPart, or FunctionToolCallEvent. It only needs events with application semantics.\n\nThis realization dictated the direction of our entire implementation.\n\nWith this principle in place, the real question became: **where exactly should this translation layer live?**\n\nIf we simply let the FastAPI route call agent.run(), parse events, format SSE, and persist data right inside the endpoint, the routing code would bloat rapidly. That is not what routes are for.\n\nScriptorDB’s approach breaks the pipeline down into three layers, each with only a single reason to change:\n\n```\nPydantic AI Event Stream        ↓agent_runner.py   → Translates framework events into application events (dicts)        ↓streaming.py      → Wraps application events into SSE strings        ↓chat.py           → HTTP lifecycle + persistence wrapping up        ↓Frontend stream.ts→ Parses SSE streams, triggers state updates        ↓useRuns.ts        → Reconstructs events into renderable Run objects\n```\n\nThe key isn’t the number of layers, but that each layer holds exactly one responsibility-meaning they can evolve independently.\n\nThis is the core of the whole design. run_agent_stream() creates an asyncio.Queue, starts agent.run() as a background task, and then intercepts Pydantic AI's raw events within event_stream_handler, translating them into our application events and pushing them into the Queue.\n\nWhy use an asyncio.Queue? Because event_stream_handler is called by Pydantic AI's internal coroutine, while the SSE response is consumed by a different coroutine. The Queue decouples the producer from the consumer, ensuring the agent isn't blocked if SSE transmission is slow.\n\nThe secret to event translation is that it’s not just “renaming.” For example, when a FunctionToolResultEvent arrives, we need to:\n\nRunTracker is a lightweight in-memory tracker that logs all metadata for a single Agent Run: which tool started when, when it ended, what the final output was, and whether the status is completed or error. It never touches the database; it is strictly maintained in memory, and once the stream concludes, chat.py converts it into a StoredRun for persistence.\n\nstream_agent_response() pulls dict events from agent_runner.py and converts most of them directly into SSE strings. However, it does two extra things:\n\nFirst, **filtering ****new_messages**. These messages are collected during the stream but are not pushed to the frontend, because the frontend doesn't need them. After the stream ends, they are passed to chat.py for persistence.\n\nSecond, **enriching model metadata**. The metadata event is supplemented with canonical_slug, display_name, and provider_specific_id. This is done so the frontend displays \"Claude 3.5 Sonnet\" rather than a cryptic internal vendor ID-echoing the Canonical Model Registry we discussed in the first article.\n\nThe FastAPI route only does three things:\n\nThe route doesn’t touch Agent details or SSE formatting; it solely handles HTTP and persistence. This is a level of coupling we are perfectly comfortable with.\n\nLooking back, what we did wasn’t just “renaming framework events and sending them to the frontend.” **We were defining an application protocol** -a contract independent of any LLM framework that simply describes “what happened during the Agent’s execution.”\n\nBecause the backend handled the heavy lifting of translation, the frontend actually ended up with less code.\n\nfrontend/src/api/stream.ts does just one thing: it reads the SSE stream, parses the event: and data: lines into a StreamRunEvent, and passes them to the higher layers via callbacks.\n\nfrontend/src/hooks/useRuns.ts maintains a reducer that incrementally builds the Run state based on the event type:\n\nThe frontend has no idea what ModelMessage, ToolReturnPart, or FunctionToolCallEvent are. It only recognizes our custom set of events, allowing it to complete state management with a single reducer. This is the direct payoff of backend translation: a much lighter frontend.\n\n**Exception Handling.** When agent.run() throws an exception internally, the consumer pulling from the Queue cannot be allowed to deadlock. We catch the exception in a try/except block and yield an error event followed by a run_end event, ensuring the frontend can properly terminate its loading state.\n\n**Event Granularity.** We unified the abstraction of all tool invocations into tool_call / tool_result, hiding the discrepancies between different providers. The cost is that some provider-specific, fine-grained events are lost; the benefit is a frontend protocol that remains stable in the long run.\n\n**Timing of Persistence.** We can’t hit the disk with every single event; data must be written in bulk after the entire run concludes. Otherwise, performance would suffer, and if a run were to fail midway, the Session would be left with an incomplete state.\n\n**Deduplicating ****new_messages.** The new_messages list returned by Pydantic AI typically includes the ModelRequest containing the user's initial input. Before saving to the Session, we check: if the first message is purely a UserPromptPart, we strip it out, because the user's message was already saved separately.\n\nThe benefits of this design are gradual, but their impact is profound:\n\n**Protocol Agnostic.** We’re using SSE today, but if we ever need to switch to WebSockets tomorrow, we’d only have to modify streaming.py. agent_runner.py and the frontend would remain completely unaffected.\n\n**Testability.** The core logic within agent_runner.py -event translation and RunTracker state updates-can be fully covered by unit tests without having to spin up an LLM.\n\n**Observability.** Every tool invocation carries its duration_ms, error_code, and trace_steps. You can inspect the run logs directly during debugging and auditing.\n\nPydantic AI’s agent.run() is designed for a \"caller.\" It assumes you only need the final result. But a frontend in a production system is an \"observer\"-it needs to see the process.\n\nThis gap cannot be bridged by the framework alone, because a framework doesn’t know what kind of product it will be embedded into. ScriptorDB’s solution isn’t to relentlessly hack the framework, but to build a translation layer just outside its boundaries: translating framework events into application events, translating application events into a transport protocol, and handing that transport protocol off to the frontend.\n\nThis translation layer isn’t specific to Pydantic AI. The same architectural boundary applies whether your backend is LangGraph, OpenAI Agents SDK, or a custom orchestrator.\n\n*Originally published at **https://aivault.dev**.*\n\n[Why I Refused to Expose Pydantic AI to My Frontend — Research Notes](https://pub.towardsai.net/why-i-refused-to-expose-pydantic-ai-to-my-frontend-research-notes-da025a8c19f7) was originally published in [Towards AI](https://pub.towardsai.net) on Medium, where people are continuing the conversation by highlighting and responding to this story.", "url": "https://wpnews.pro/news/why-i-refused-to-expose-pydantic-ai-to-my-frontend-research-notes", "canonical_source": "https://pub.towardsai.net/why-i-refused-to-expose-pydantic-ai-to-my-frontend-research-notes-da025a8c19f7?source=rss----98111c9905da---4", "published_at": "2026-07-09 12:31:00+00:00", "updated_at": "2026-07-09 12:44:00.028480+00:00", "lang": "en", "topics": ["ai-tools", "developer-tools", "ai-infrastructure"], "entities": ["Pydantic AI", "ScriptorDB"], "alternates": {"html": "https://wpnews.pro/news/why-i-refused-to-expose-pydantic-ai-to-my-frontend-research-notes", "markdown": "https://wpnews.pro/news/why-i-refused-to-expose-pydantic-ai-to-my-frontend-research-notes.md", "text": "https://wpnews.pro/news/why-i-refused-to-expose-pydantic-ai-to-my-frontend-research-notes.txt", "jsonld": "https://wpnews.pro/news/why-i-refused-to-expose-pydantic-ai-to-my-frontend-research-notes.jsonld"}}