# How I Built a Unified API Gateway for 200+ AI Models (Architecture Deep Dive)

> Source: <https://dev.to/sarangai_id/how-i-built-a-unified-api-gateway-for-200-ai-models-architecture-deep-dive-3559>
> Published: 2026-10-06 20:37:16+00:00

Every developer who has worked with more than one AI provider knows the pain:

If you want to use 4 different models, you have to manage 4 accounts, 4 invoices, and 4 sets of documentation. This isn't just annoying — it's a **bottleneck** that stops developers from experimenting with new models.

I built **SarangAI** to solve this. It's a unified AI gateway that routes all requests to 200+ models through a **single OpenAI-compatible endpoint**.

In this article, I'll walk through the architecture behind it, the design decisions I made, and the technical challenges that came up.

At a high level, SarangAI consists of 4 main components:

The flow is simple:

```
Client → API Gateway → Router → Provider Adapter → AI Provider
                ↓
          Billing & Rate Limiter
```

This is the most important design decision I made.

When I started building SarangAI, I had two options:

**Option 1:** Build my own API format, my own docs, my own SDK.

**Option 2:** Use the OpenAI format, which has become the de-facto standard.

I chose **option 2**. Here's why:

This is what makes SarangAI usable in minutes, not hours.

Every provider has a different response format. OpenAI has `choices[0].message.content`, Anthropic has `content[0].text`, Google has yet another structure.

The solution: **adapter pattern**. Each provider has an adapter that:

This keeps client-side code clean — they don't need to know which provider is being used.

Streaming responses are tricky. Every provider sends chunks differently:

`data: {...}`
`content_block_delta`, etc.)
In SarangAI, I normalize all streaming to the **same SSE format** as OpenAI. So clients only need to handle one streaming format.

Since SarangAI uses a **prepaid IDR top-up** model, I need to:

For this, I use a combination of **Redis** (for fast balance checks) and a **database** (for audit trails).

One of SarangAI's main features is **instant model switching**. Users can change models without restarting their app.

This means the router has to:

I made this router **stateless**, so it can scale horizontally without issues.

`sarangai-cli`
Besides the API gateway, I also built a CLI tool that works directly from the terminal:

```
npm install -g sarangai-cli
sarang
```

The CLI connects to the SarangAI endpoint and gives you an interactive workspace. You can:

This is especially useful for developers who live in the terminal.

**1. Standards matter.**

Choosing the OpenAI-compatible format was the best decision I made. It's what makes adoption fast.

**2. Adapter patterns save lives.**

Without clean adapters, adding a new provider would be a nightmare.

**3. Prepaid > Subscription for developer tools.**

Developers hate monthly subscriptions. Prepaid gives them a sense of full control.

**4. Documentation is a feature.**

No matter how good your architecture is, if the docs are bad, nobody will use it.

If you work with multiple AI models regularly, give SarangAI a try:

Install the CLI:

```
npm install -g sarangai-cli
```

I'm curious: **what's your current setup for handling multiple AI providers?** Do you use a library? Or manage them one by one?

Share in the comments - I'd love to hear how other developers handle this.