{"slug": "multi-provider-llm-router-or-how-i-got-tired-of-forgetting-which-api-format-i-to", "title": "Multi-Provider LLM Router, or How I Got Tired of Forgetting Which API Format I Had To Use", "summary": "A developer has open-sourced fastapi-multi-llm-starter, an MIT-licensed FastAPI template that unifies streaming access to multiple LLM providers including Anthropic, Google, OpenAI, and DeepSeek behind a single asynchronous endpoint. The router exposes POST /v1/chat/stream, which accepts a unified payload and emits standardized SSE events for thinking, content, tool calls, and completion, with models defined dynamically in a models.json catalog rather than hardcoded. The repository includes pytest coverage, Pydantic v2 schemas, and a built-in testing playground at localhost:8000.", "body_md": "If you've ever built an application that integrates with multiple LLM providers (Anthropic, Google, OpenAI, DeepSeek), you already know the pain:\n\nI recently extracted the core streaming router from my platform into an open-source FastAPI template. Here is how it works.\n\nA single asynchronous endpoint:\n\n`POST /v1/chat/stream`\n\nIt accepts a unified request payload and returns a standardized SSE stream emitting four clean events:\n\n`event: thinking` — Internal model reasoning tokens (streamed in real-time).`event: content` — User-facing response text.`event: tool_call` — Function calling requests.`event: done` — Stream completion (`[DONE]`).\nInstead of pulling heavy wrapper frameworks, use direct asynchronous HTTP via `httpx.AsyncClient` and the official Google GenAI SDK:\n\n```\nfastapi-multi-llm-starter/\n├── app/\n│   ├── config.py       # Pydantic Settings loading environment variables\n│   ├── main.py         # FastAPI app with CORS, health check & test playground\n│   ├── models.json     # Dynamic model catalog (Claude, Gemini, GPT)\n│   ├── router.py       # Unified multi-provider async stream dispatcher\n│   └── schemas.py      # Strict Pydantic v2 validation models\n├── tests/              # Automated unit tests (pytest)\n├── requirements.txt\n└── README.md\n```\n\nI disliked the idea of hardcoded models, so I decoupled them into a `models.json` file:\n\n```\n{\n  \"models\": [\n    {\n      \"id\": \"claude-sonnet-5\",\n      \"name\": \"Claude Sonnet 5\",\n      \"provider\": \"Anthropic\",\n      \"thinking\": true\n    },\n    {\n      \"id\": \"gemini-3.8-flash\",\n      \"name\": \"Gemini 3.8 Flash\",\n      \"provider\": \"Google\",\n      \"thinking\": true\n    },\n    {\n      \"id\": \"gpt-5.6-terra\",\n      \"name\": \"GPT 5.6 Terra\",\n      \"provider\": \"OpenAI\",\n      \"thinking\": true\n    }\n  ]\n}\n```\n\nNow, if you want to add another model, you just edit the JSON. The backend and the embedded UI dynamically populate available models via `GET /v1/models`.\n\nThe repository includes a testing playground running directly at `http://localhost:8000/`. You can immediately test prompts, check streaming latency, and verify reasoning blocks without setting up a frontend framework.\n\nOf course, you'll need your own API keys.\n\nThe full core code is open-source under the **MIT License** on GitHub:\n\n👉 [github.com/wolfnomknight/fastapi-multi-llm-starter](https://github.com/wolfnomknight/fastapi-multi-llm-starter)\n\nIncludes full `pytest` test coverage, `.env.example`, and clean Pydantic v2 schemas.\n\nFeel free to fork it, use it in your side projects or micro-SaaS, and let me know if you run into any issues or have ideas for additional providers!", "url": "https://wpnews.pro/news/multi-provider-llm-router-or-how-i-got-tired-of-forgetting-which-api-format-i-to", "canonical_source": "https://dev.to/wolfnom/multi-provider-llm-router-or-how-i-got-tired-of-forgetting-which-api-format-i-had-to-use-lk3", "published_at": "2026-09-10 03:53:49+00:00", "updated_at": "2026-09-10 04:20:26.859472+00:00", "lang": "en", "topics": ["large-language-models", "ai-tools", "developer-tools", "ai-infrastructure", "ai-agents"], "entities": ["FastAPI", "Anthropic", "Google", "OpenAI", "DeepSeek", "GitHub", "Pydantic", "httpx"], "alternates": {"html": "https://wpnews.pro/news/multi-provider-llm-router-or-how-i-got-tired-of-forgetting-which-api-format-i-to", "markdown": "https://wpnews.pro/news/multi-provider-llm-router-or-how-i-got-tired-of-forgetting-which-api-format-i-to.md", "text": "https://wpnews.pro/news/multi-provider-llm-router-or-how-i-got-tired-of-forgetting-which-api-format-i-to.txt", "jsonld": "https://wpnews.pro/news/multi-provider-llm-router-or-how-i-got-tired-of-forgetting-which-api-format-i-to.jsonld"}}