The AI API Will Fail on You — Here's How I Structure FastAPI to Handle It

A developer shared a FastAPI pattern for handling failures when integrating AI APIs, emphasizing thin routers, a service layer with typed exceptions, and Pydantic validation of AI responses. The approach maps provider errors to appropriate HTTP status codes, improving reliability in production.

A few weeks ago I was demoing a small FastAPI service that called an LLM to summarize text. Worked great in every test I ran. Then, live, the provider API took almost 9 seconds to respond and my endpoint just... hung. No error, no timeout, just silence while I sat there making small talk to fill the gap. That was the moment it clicked for me: when you put an AI API behind your own API, you've inherited two sets of failure modes — yours and theirs. Rate limits, timeouts, malformed JSON in the response, the model deciding to add a chatty preamble before the JSON you asked for. None of that is hypothetical. It's Tuesday. Here's how I've learned to structure a FastAPI backend so that when not if the AI call misbehaves, it fails in a way you control. The most common pattern I see and the one I used to write is catching exceptions right inside the endpoint function. It works until you have more than one endpoint calling the model, and suddenly your error handling is copy-pasted five times with five slightly different bugs. Instead, the router's only job is to receive the request, validate it, and hand it off: python @router.post "/summarize" async def summarize payload: SummarizeRequest, service: SummaryService = Depends : result = await service.run payload return result All the "what could go wrong" logic lives one layer down, in the service. Generic except Exception blocks tell you nothing about why something failed, which makes writing a sane HTTP response back to your client impossible. I define a small hierarchy instead: class AIProviderError Exception : """Base error for anything the model provider throws at us.""" class AIProviderTimeout AIProviderError : pass class AIProviderRateLimited AIProviderError : pass class AIResponseMalformed AIProviderError : """The provider responded, but not with what we asked for.""" The service layer catches the provider SDK's raw exceptions and re-raises them as one of these. A dedicated exception handler at the app level then maps each one to the right HTTP status: 504 for a timeout, 429 for rate limits, 502 when the model's output doesn't parse. Your client gets something actionable instead of a raw 500 and a stack trace. This is the one people skip. We're all trained to validate incoming requests with Pydantic, but the response coming back from the AI is just as untrusted. Models occasionally return almost-JSON, or valid JSON with a field renamed, or an extra sentence wrapped around it. class SummaryResult BaseModel : summary: str key points: list str def parse model output raw: str - SummaryResult: try: data = json.loads raw return SummaryResult.model validate data except json.JSONDecodeError, ValidationError as e: raise AIResponseMalformed str e from e If it doesn't fit the schema, it doesn't get past this line — full stop. That one function has saved me from shipping garbage data downstream more times than I'd like to admit. None of this is complicated once you see it laid out, but I didn't see it laid out anywhere when I needed it — I pieced it together across a few projects, mostly the hard way, after enough live demos went sideways. Routers that stay thin, a service layer that owns the provider calls, a typed exception hierarchy, and Pydantic validating both directions instead of just one. I ended up packaging exactly this structure — routers, services, schemas, and the error-handling layer between your FastAPI app and an AI model — into a small starter template so I'd stop rebuilding it from scratch every time. If you want the full working reference instead of piecing it together yourself, you can grab it here https://saljazz5.gumroad.com/l/ujogq . Worth noting: it's a structural pattern, not a full production app — no JWT auth or database included, on purpose, so it stays easy to read and adapt to whatever you're building. I'm also currently open to backend/full-stack roles Python, FastAPI, Flask, AI integrations and freelance work — feel free to check out more of my code on GitHub https://github.com/GerAle30 .