# Multi-Provider LLM Router, or How I Got Tired of Forgetting Which API Format I Had To Use

> Source: <https://dev.to/wolfnom/multi-provider-llm-router-or-how-i-got-tired-of-forgetting-which-api-format-i-had-to-use-lk3>
> Published: 2026-09-10 03:53:49+00:00

If you've ever built an application that integrates with multiple LLM providers (Anthropic, Google, OpenAI, DeepSeek), you already know the pain:

I recently extracted the core streaming router from my platform into an open-source FastAPI template. Here is how it works.

A single asynchronous endpoint:

`POST /v1/chat/stream`

It accepts a unified request payload and returns a standardized SSE stream emitting four clean events:

`event: thinking` — Internal model reasoning tokens (streamed in real-time).`event: content` — User-facing response text.`event: tool_call` — Function calling requests.`event: done` — Stream completion (`[DONE]`).
Instead of pulling heavy wrapper frameworks, use direct asynchronous HTTP via `httpx.AsyncClient` and the official Google GenAI SDK:

```
fastapi-multi-llm-starter/
├── app/
│   ├── config.py       # Pydantic Settings loading environment variables
│   ├── main.py         # FastAPI app with CORS, health check & test playground
│   ├── models.json     # Dynamic model catalog (Claude, Gemini, GPT)
│   ├── router.py       # Unified multi-provider async stream dispatcher
│   └── schemas.py      # Strict Pydantic v2 validation models
├── tests/              # Automated unit tests (pytest)
├── requirements.txt
└── README.md
```

I disliked the idea of hardcoded models, so I decoupled them into a `models.json` file:

```
{
  "models": [
    {
      "id": "claude-sonnet-5",
      "name": "Claude Sonnet 5",
      "provider": "Anthropic",
      "thinking": true
    },
    {
      "id": "gemini-3.8-flash",
      "name": "Gemini 3.8 Flash",
      "provider": "Google",
      "thinking": true
    },
    {
      "id": "gpt-5.6-terra",
      "name": "GPT 5.6 Terra",
      "provider": "OpenAI",
      "thinking": true
    }
  ]
}
```

Now, if you want to add another model, you just edit the JSON. The backend and the embedded UI dynamically populate available models via `GET /v1/models`.

The repository includes a testing playground running directly at `http://localhost:8000/`. You can immediately test prompts, check streaming latency, and verify reasoning blocks without setting up a frontend framework.

Of course, you'll need your own API keys.

The full core code is open-source under the **MIT License** on GitHub:

👉 [github.com/wolfnomknight/fastapi-multi-llm-starter](https://github.com/wolfnomknight/fastapi-multi-llm-starter)

Includes full `pytest` test coverage, `.env.example`, and clean Pydantic v2 schemas.

Feel free to fork it, use it in your side projects or micro-SaaS, and let me know if you run into any issues or have ideas for additional providers!