Multi-Provider LLM Router, or How I Got Tired of Forgetting Which API Format I Had To Use

A developer has open-sourced fastapi-multi-llm-starter, an MIT-licensed FastAPI template that unifies streaming access to multiple LLM providers including Anthropic, Google, OpenAI, and DeepSeek behind a single asynchronous endpoint. The router exposes POST /v1/chat/stream, which accepts a unified payload and emits standardized SSE events for thinking, content, tool calls, and completion, with models defined dynamically in a models.json catalog rather than hardcoded. The repository includes pytest coverage, Pydantic v2 schemas, and a built-in testing playground at localhost:8000.

If you've ever built an application that integrates with multiple LLM providers Anthropic, Google, OpenAI, DeepSeek , you already know the pain: I recently extracted the core streaming router from my platform into an open-source FastAPI template. Here is how it works. A single asynchronous endpoint: POST /v1/chat/stream It accepts a unified request payload and returns a standardized SSE stream emitting four clean events: event: thinking — Internal model reasoning tokens streamed in real-time . event: content — User-facing response text. event: tool call — Function calling requests. event: done — Stream completion DONE . Instead of pulling heavy wrapper frameworks, use direct asynchronous HTTP via httpx.AsyncClient and the official Google GenAI SDK: fastapi-multi-llm-starter/ ├── app/ │ ├── config.py Pydantic Settings loading environment variables │ ├── main.py FastAPI app with CORS, health check & test playground │ ├── models.json Dynamic model catalog Claude, Gemini, GPT │ ├── router.py Unified multi-provider async stream dispatcher │ └── schemas.py Strict Pydantic v2 validation models ├── tests/ Automated unit tests pytest ├── requirements.txt └── README.md I disliked the idea of hardcoded models, so I decoupled them into a models.json file: { "models": { "id": "claude-sonnet-5", "name": "Claude Sonnet 5", "provider": "Anthropic", "thinking": true }, { "id": "gemini-3.8-flash", "name": "Gemini 3.8 Flash", "provider": "Google", "thinking": true }, { "id": "gpt-5.6-terra", "name": "GPT 5.6 Terra", "provider": "OpenAI", "thinking": true } } Now, if you want to add another model, you just edit the JSON. The backend and the embedded UI dynamically populate available models via GET /v1/models . The repository includes a testing playground running directly at http://localhost:8000/ . You can immediately test prompts, check streaming latency, and verify reasoning blocks without setting up a frontend framework. Of course, you'll need your own API keys. The full core code is open-source under the MIT License on GitHub: 👉 github.com/wolfnomknight/fastapi-multi-llm-starter https://github.com/wolfnomknight/fastapi-multi-llm-starter Includes full pytest test coverage, .env.example , and clean Pydantic v2 schemas. Feel free to fork it, use it in your side projects or micro-SaaS, and let me know if you run into any issues or have ideas for additional providers