{"slug": "the-ai-api-will-fail-on-you-here-s-how-i-structure-fastapi-to-handle-it", "title": "The AI API Will Fail on You — Here's How I Structure FastAPI to Handle It", "summary": "A developer shared a FastAPI pattern for handling failures when integrating AI APIs, emphasizing thin routers, a service layer with typed exceptions, and Pydantic validation of AI responses. The approach maps provider errors to appropriate HTTP status codes, improving reliability in production.", "body_md": "A few weeks ago I was demoing a small FastAPI service that called an LLM to summarize text. Worked great in every test I ran. Then, live, the provider API took almost 9 seconds to respond and my endpoint just... hung. No error, no timeout, just silence while I sat there making small talk to fill the gap.\n\nThat was the moment it clicked for me: when you put an AI API behind your own API, you've inherited *two* sets of failure modes — yours and theirs. Rate limits, timeouts, malformed JSON in the response, the model deciding to add a chatty preamble before the JSON you asked for. None of that is hypothetical. It's Tuesday.\n\nHere's how I've learned to structure a FastAPI backend so that when (not if) the AI call misbehaves, it fails in a way you control.\n\nThe most common pattern I see (and the one I used to write) is catching exceptions right inside the endpoint function. It works until you have more than one endpoint calling the model, and suddenly your error handling is copy-pasted five times with five slightly different bugs.\n\nInstead, the router's only job is to receive the request, validate it, and hand it off:\n\n``` python\n@router.post(\"/summarize\")\nasync def summarize(payload: SummarizeRequest, service: SummaryService = Depends()):\n    result = await service.run(payload)\n    return result\n```\n\nAll the \"what could go wrong\" logic lives one layer down, in the service.\n\nGeneric `except Exception`\n\nblocks tell you nothing about *why* something failed, which makes writing a sane HTTP response back to your client impossible. I define a small hierarchy instead:\n\n```\nclass AIProviderError(Exception):\n    \"\"\"Base error for anything the model provider throws at us.\"\"\"\n\nclass AIProviderTimeout(AIProviderError):\n    pass\n\nclass AIProviderRateLimited(AIProviderError):\n    pass\n\nclass AIResponseMalformed(AIProviderError):\n    \"\"\"The provider responded, but not with what we asked for.\"\"\"\n```\n\nThe service layer catches the provider SDK's raw exceptions and re-raises them as one of these. A dedicated exception handler at the app level then maps each one to the right HTTP status: 504 for a timeout, 429 for rate limits, 502 when the model's output doesn't parse. Your client gets something actionable instead of a raw 500 and a stack trace.\n\nThis is the one people skip. We're all trained to validate incoming requests with Pydantic, but the response coming *back* from the AI is just as untrusted. Models occasionally return almost-JSON, or valid JSON with a field renamed, or an extra sentence wrapped around it.\n\n```\nclass SummaryResult(BaseModel):\n    summary: str\n    key_points: list[str]\n\ndef parse_model_output(raw: str) -> SummaryResult:\n    try:\n        data = json.loads(raw)\n        return SummaryResult.model_validate(data)\n    except (json.JSONDecodeError, ValidationError) as e:\n        raise AIResponseMalformed(str(e)) from e\n```\n\nIf it doesn't fit the schema, it doesn't get past this line — full stop. That one function has saved me from shipping garbage data downstream more times than I'd like to admit.\n\nNone of this is complicated once you see it laid out, but I didn't see it laid out anywhere when I needed it — I pieced it together across a few projects, mostly the hard way, after enough live demos went sideways. Routers that stay thin, a service layer that owns the provider calls, a typed exception hierarchy, and Pydantic validating both directions instead of just one.\n\nI ended up packaging exactly this structure — routers, services, schemas, and the error-handling layer between your FastAPI app and an AI model — into a small starter template so I'd stop rebuilding it from scratch every time. If you want the full working reference instead of piecing it together yourself, you can grab it [here](https://saljazz5.gumroad.com/l/ujogq). Worth noting: it's a structural pattern, not a full production app — no JWT auth or database included, on purpose, so it stays easy to read and adapt to whatever you're building.\n\nI'm also currently open to backend/full-stack roles (Python, FastAPI, Flask, AI integrations) and freelance work — feel free to check out more of my code on [GitHub](https://github.com/GerAle30).", "url": "https://wpnews.pro/news/the-ai-api-will-fail-on-you-here-s-how-i-structure-fastapi-to-handle-it", "canonical_source": "https://dev.to/gerale30/the-ai-api-will-fail-on-you-heres-how-i-structure-fastapi-to-handle-it-53ia", "published_at": "2026-08-28 17:25:35+00:00", "updated_at": "2026-08-28 17:50:14.599273+00:00", "lang": "en", "topics": ["developer-tools", "ai-products", "ai-tools"], "entities": ["FastAPI", "Pydantic", "AIProviderError", "SummaryService", "SummaryResult"], "alternates": {"html": "https://wpnews.pro/news/the-ai-api-will-fail-on-you-here-s-how-i-structure-fastapi-to-handle-it", "markdown": "https://wpnews.pro/news/the-ai-api-will-fail-on-you-here-s-how-i-structure-fastapi-to-handle-it.md", "text": "https://wpnews.pro/news/the-ai-api-will-fail-on-you-here-s-how-i-structure-fastapi-to-handle-it.txt", "jsonld": "https://wpnews.pro/news/the-ai-api-will-fail-on-you-here-s-how-i-structure-fastapi-to-handle-it.jsonld"}}