The first time we added an AI feature to a client's Django app, we did it the naive way: POST request, wait, get a response, render it. It worked. It also felt broken. Users stared at a spinner for 8 seconds, then got a wall of text. They stopped using it. Not because the AI was wrong — it wasn't — but because waiting for the full response before showing anything felt laggy in a way that ChatGPT had trained them out of tolerating.
Streaming solves this. It's not just a UX nicety — users start reading before the response finishes, they feel like something is happening, and they're more likely to catch when the AI goes off the rails early and stop it. We've implemented streaming across a handful of production Django+React applications now. Here's the full stack: server-side, client-side, and the edge cases that'll catch you if you skip the boring infrastructure bits.
WebSockets are overkill for LLM streaming. The communication is one-directional — the server sends chunks, the client reads them. Server-Sent Events (SSE) is the right primitive here: it's HTTP, it works through proxies, and Django supports it without any extra infrastructure.
The core view uses Django's StreamingHttpResponse
with a generator that yields tokens as they arrive from the LLM API:
import json
import anthropic
from django.http import StreamingHttpResponse
from django.views.decorators.csrf import csrf_exempt
from django.views.decorators.http import require_POST
client = anthropic.Anthropic()
def stream_llm_response(prompt: str, system: str = ""):
"""Generator that yields SSE-formatted chunks from Claude."""
with client.messages.stream(
model="claude-opus-4-5",
max_tokens=1024,
system=system,
messages=[{"role": "user", "content": prompt}],
) as stream:
for text_chunk in stream.text_stream:
payload = json.dumps({"chunk": text_chunk, "done": False})
yield f"data: {payload}\n\n"
yield f"data: {json.dumps({'chunk': '', 'done': True})}\n\n"
@require_POST
@csrf_exempt # Handle CSRF separately — see note below
def ai_stream_view(request):
body = json.loads(request.body)
prompt = body.get("prompt", "").strip()
if not prompt:
return JsonResponse({"error": "Prompt is required"}, status=400)
response = StreamingHttpResponse(
stream_llm_response(prompt),
content_type="text/event-stream",
)
response["Cache-Control"] = "no-cache"
response["X-Accel-Buffering"] = "no" # Disable Nginx buffering
return response
The X-Accel-Buffering: no
header is important. If you're running behind Nginx (you are in production), it buffers responses by default. Without that header, your users won't see any streaming — they'll get the whole response at once after the stream completes, which defeats the entire purpose.
On the CSRF note: SSE endpoints receive POST requests with JSON bodies, which means standard Django CSRF tokens work fine — you just need to include the token in your request headers. We use @csrf_exempt
during development and add a custom middleware-level check in production that validates the token from the X-CSRFToken
header.
EventSource
is the browser API for consuming SSE. It's available in every modern browser without any dependencies:
import { useState, useCallback } from "react";
function useStreamingChat() {
const [response, setResponse] = useState("");
const [isStreaming, setIsStreaming] = useState(false);
const [error, setError] = useState<string | null>(null);
const sendPrompt = useCallback(async (prompt: string) => {
setResponse("");
setError(null);
setIsStreaming(true);
try {
// POST the prompt first to get a stream token (see auth section)
const initRes = await fetch("/api/ai/stream/", {
method: "POST",
headers: {
"Content-Type": "application/json",
"X-CSRFToken": getCsrfToken(),
},
body: JSON.stringify({ prompt }),
});
if (!initRes.ok) throw new Error("Failed to initialise stream");
if (!initRes.body) throw new Error("No response body");
// Use ReadableStream directly — more control than EventSource
const reader = initRes.body.getReader();
const decoder = new TextDecoder();
let buffer = "";
while (true) {
const { done, value } = await reader.read();
if (done) break;
buffer += decoder.decode(value, { stream: true });
const lines = buffer.split("\n\n");
buffer = lines.pop() ?? "";
for (const line of lines) {
if (!line.startsWith("data: ")) continue;
const data = JSON.parse(line.slice(6));
if (data.done) {
setIsStreaming(false);
return;
}
setResponse((prev) => prev + data.chunk);
}
}
} catch (err) {
setError(err instanceof Error ? err.message : "Stream failed");
setIsStreaming(false);
}
}, []);
return { response, isStreaming, error, sendPrompt };
}
function getCsrfToken(): string {
return document.cookie
.split("; ")
.find((row) => row.startsWith("csrftoken="))
?.split("=")[1] ?? "";
}
We use fetch
with a ReadableStream
reader rather than EventSource
for two reasons. First, EventSource
only supports GET requests, which means you can't POST a JSON body — you'd have to encode the prompt as a query parameter (ugly, limited by URL length, logged in server access logs). Second, ReadableStream
gives you more control over the buffer and parsing logic.
The progressive render is just React state: every chunk appended to the response string, which re-renders the text as it arrives. We pair it with a simple markdown renderer so the formatting comes through correctly as the text streams in.
Three things will break your streaming in production that work perfectly in development:
Gunicorn workers. Gunicorn's default worker type (sync
) holds the worker thread for the entire duration of the stream. If you have 4 workers and 4 concurrent streams, your app is completely blocked. Switch to gevent
or gthread
workers for streaming endpoints, or run the stream endpoint on a separate Gunicorn instance with async workers.
Load balancer timeouts. AWS ALB and most load balancers have a default idle timeout of 60 seconds. A slow LLM response generating a long answer can easily hit this. Set your load balancer's idle timeout to at least 300 seconds on paths that serve streaming responses.
Celery integration. If your AI calls go through Celery tasks (which they often should, for queueing and retry logic), you can't stream directly from a task return value. The pattern we use: the task writes chunks to Redis as they arrive, and a thin Django view streams from Redis to the client using the same SSE pattern. Slightly more moving parts but it decouples the AI call from the HTTP connection lifecycle.
Streaming makes latency feel lower — it doesn't reduce it. Your time-to-first-token is still whatever it is, which can be 1-3 seconds for large models. If that matters for your use case (real-time autocomplete, for example), streaming is the wrong solution — you need a faster or smaller model.
Streaming also doesn't help with error recovery mid-stream. If the LLM call fails after sending 50 tokens, you've already sent those tokens to the client and committed to a partial response. We handle this with a reconnect protocol: the client stores the partial response, and if the stream dies, it can restart with context about what was already received. Most of the time we just show the partial response with a "response interrupted" notice and let the user re-run — that's good enough for the use cases we've shipped.
Finally, streaming adds complexity to your response logging. You can't log the full response until the stream completes, which means your request log entry closes before the response is done. We use a post-stream hook that fires after the generator exhausts to write the full response to the database.
Streaming LLM responses in Django + React is not complicated once you have the right mental model: SSE on the server, ReadableStream
on the client, and careful attention to the infrastructure layer that sits in between. The actual API integration is about 30 lines of code. The production-readiness — Gunicorn workers, Nginx headers, load balancer timeouts — takes longer to get right than the feature itself.
We've shipped this pattern across Django monoliths, Django + React SPAs, and Django as a pure API backend. The core implementation is the same each time. What changes is the infrastructure configuration, and that's the part worth spending time on.
Lycore builds production AI systems for businesses — we implement streaming AI interfaces, full-stack Django + React applications, and production-ready LLM integrations that hold up under real user load. Get in touch if you want to talk through your use case.