# Streaming an AI agent's tool calls into a Next.js UI: the FastAPI pattern I keep reusing

> Source: <https://dev.to/cpcai_studio/streaming-an-ai-agents-tool-calls-into-a-nextjs-ui-the-fastapi-pattern-i-keep-reusing-1dg7>
> Published: 2026-11-18 08:11:18+00:00

**Recommendation:** Emit agent tool calls as structured SSE events from a queue rather than calling the model in the HTTP handler, and buffer partial frames client-side so chunk boundaries don't break parsing.
More in the playbook "Designing agent systems": <https://wpnews.pro/playbooks/agent-architecture>

Most AI app prototypes I get handed are stuck on the same screen: the chat box shows a spinner, then a fully formed paragraph appears. Under the hood the agent may have called four tools, waited thirty seconds and retried twice, but the person using it sees nothing until the end. On a slow model that reads as a broken app, and people close the tab before the answer lands.

Streaming the agent's steps, not just its tokens, fixes most of that. Here is the shape I use for FastAPI plus Next.js builds, with the parts that actually broke in production.

Before any code, decide what the client is allowed to see. I keep four event types and nothing else:

`token`: a piece of assistant text` tool_start`: {"name": "...", "args": {...}}` tool_end`: {"name": "...", "ok": true, "ms": 812, "summary": "..."}` done`: {"finish_reason": "stop"}
Everything the UI renders comes from one of these. That constraint is what stops the front end from turning into a pile of special cases, and it makes replaying a run trivial later: you store the events, not the rendered HTML.

The endpoint does not call the model directly; it yields events from a queue that the agent runner writes to. That separation matters because tool calls happen inside the agent loop, often in a thread, and you do not want the HTTP handler to know about them.

``` python
import asyncio, json
from fastapi import FastAPI
from fastapi.responses import StreamingResponse

app = FastAPI()

async def run_agent(prompt: str, q: asyncio.Queue):
    try:
        async for ev in agent_loop(prompt):          # yields dicts
            await q.put(ev)
    except Exception as e:
        await q.put({"type": "error", "message": str(e)[:200]})
    finally:
        await q.put(None)

@app.post("/api/agent")async def agent(payload: dict):
    q: asyncio.Queue = asyncio.Queue()

    async def gen():
        asyncio.create_task(run_agent(payload["prompt"], q))
        while True:
            ev = await q.get()
            if ev is None:
                break
            yield f"data: {json.dumps(ev, ensure_ascii=False)}\n\n"

    return StreamingResponse(
        gen(),
        media_type="text/event-stream",
        headers={"Cache-Control": "no-cache", "X-Accel-Buffering": "no"},
    )
```

Two headers there are not optional. Without `X-Accel-Buffering: no`, a proxy may buffer the whole stream and the user gets the spinner experience anyway. Without `Cache-Control: no-cache`, some CDNs will happily cache a partial response.

Do not put blocking calls inside an async generator. If your model SDK is synchronous, wrap the call with `asyncio.to_thread` and push into the queue from there; a blocking loop inside the generator stalls the whole worker for every other request it is serving.

`EventSource` only does GET, so for a POST body I use `fetch` with a reader:

``` js
export async function streamAgent(prompt, onEvent) {
  const res = await fetch("/api/agent", {
    method: "POST",
    headers: { "Content-Type": "application/json" },
    body: JSON.stringify({ prompt }),
  });
  if (!res.body) throw new Error("no stream");

  const reader = res.body.getReader();
  const dec = new TextDecoder();
  let buf = "";

  while (true) {
    const { value, done } = await reader.read();
    if (done) break;
    buf += dec.decode(value, { stream: true });
    const parts = buf.split("\n\n");
    buf = parts.pop() ?? "";
    for (const p of parts) {
      const line = p.split("\n").find((l) => l.startsWith("data: "));
      if (line) onEvent(JSON.parse(line.slice(6)));
    }
  }
}
```

The buffering detail (`buf`, `split("\n\n")`, keep the remainder) is where these break. A chunk boundary can land in the middle of a JSON object, and `JSON.parse` on a partial object throws once, then the handler is gone and the stream silently stops. Keep the leftover in the buffer and parse only complete frames.

**Idempotency.** A user double-clicking send, or a client reconnecting, can start the same run twice. Send a client-generated `run_id` and have the server treat a repeat within a short window as a lookup rather than a new run. Cheap to add, expensive to debug later.

**Cancellation.** When the tab closes, the generator is cancelled, but the agent task keeps burning tokens unless you cancel it. Hold the task reference and cancel it in a `finally` block, or run the agent under a timeout shorter than the user's patience.

**Persistence of events, not text.** Store the event list per run so a refresh can rebuild the transcript, including which tools ran and how long they took. Users read "searched pricing, 1.2s" as evidence the app is working, and it costs you one table.

If this is a Supabase-backed app, the quickest way to make the prototype "work" is to turn off row level security or grant table access to the `anon` role. Do not do that. An `anon` grant makes every row readable by anyone who opens the network tab, including other users' conversations. Keep RLS enabled on every table that holds user data, write policies that bind rows to `auth.uid()`, and let the API use the service role key on the server only, never in the browser bundle. If a policy is missing you get empty results instead of private data, which is the failure you want.

With the event contract in place the UI work becomes small: a step list that appends on `tool_start` and closes on `tool_end`, text that appends on `token`. Users can see the agent working, cancel it, and come back to it after a refresh. In a five-day build, that is usually the difference between a demo that gets shared and one that gets closed.

I build these agent backends in Python and TypeScript from Seoul, usually finishing a half-built Lovable, Bolt or Cursor project in three to five days. If you have one that works locally and dies in production, the studio page is at cpcai-studio.netlify.app/en and email works too: [yyhak12345@gmail.com](mailto:yyhak12345@gmail.com)
