{"slug": "how-i-built-a-unified-api-gateway-for-200-ai-models-architecture-deep-dive", "title": "How I Built a Unified API Gateway for 200+ AI Models (Architecture Deep Dive)", "summary": "A developer built SarangAI, a unified AI gateway that routes requests to more than 200 models through a single OpenAI-compatible endpoint. The system uses a stateless router, per-provider adapters that normalize responses and streaming to OpenAI's SSE format, and a prepaid IDR billing layer backed by Redis for fast balance checks and a database for audit trails, plus a companion sarangai-cli terminal tool.", "body_md": "Every developer who has worked with more than one AI provider knows the pain:\n\nIf you want to use 4 different models, you have to manage 4 accounts, 4 invoices, and 4 sets of documentation. This isn't just annoying — it's a **bottleneck** that stops developers from experimenting with new models.\n\nI built **SarangAI** to solve this. It's a unified AI gateway that routes all requests to 200+ models through a **single OpenAI-compatible endpoint**.\n\nIn this article, I'll walk through the architecture behind it, the design decisions I made, and the technical challenges that came up.\n\nAt a high level, SarangAI consists of 4 main components:\n\nThe flow is simple:\n\n```\nClient → API Gateway → Router → Provider Adapter → AI Provider\n                ↓\n          Billing & Rate Limiter\n```\n\nThis is the most important design decision I made.\n\nWhen I started building SarangAI, I had two options:\n\n**Option 1:** Build my own API format, my own docs, my own SDK.\n\n**Option 2:** Use the OpenAI format, which has become the de-facto standard.\n\nI chose **option 2**. Here's why:\n\nThis is what makes SarangAI usable in minutes, not hours.\n\nEvery provider has a different response format. OpenAI has `choices[0].message.content`, Anthropic has `content[0].text`, Google has yet another structure.\n\nThe solution: **adapter pattern**. Each provider has an adapter that:\n\nThis keeps client-side code clean — they don't need to know which provider is being used.\n\nStreaming responses are tricky. Every provider sends chunks differently:\n\n`data: {...}`\n`content_block_delta`, etc.)\nIn SarangAI, I normalize all streaming to the **same SSE format** as OpenAI. So clients only need to handle one streaming format.\n\nSince SarangAI uses a **prepaid IDR top-up** model, I need to:\n\nFor this, I use a combination of **Redis** (for fast balance checks) and a **database** (for audit trails).\n\nOne of SarangAI's main features is **instant model switching**. Users can change models without restarting their app.\n\nThis means the router has to:\n\nI made this router **stateless**, so it can scale horizontally without issues.\n\n`sarangai-cli`\nBesides the API gateway, I also built a CLI tool that works directly from the terminal:\n\n```\nnpm install -g sarangai-cli\nsarang\n```\n\nThe CLI connects to the SarangAI endpoint and gives you an interactive workspace. You can:\n\nThis is especially useful for developers who live in the terminal.\n\n**1. Standards matter.**\n\nChoosing the OpenAI-compatible format was the best decision I made. It's what makes adoption fast.\n\n**2. Adapter patterns save lives.**\n\nWithout clean adapters, adding a new provider would be a nightmare.\n\n**3. Prepaid > Subscription for developer tools.**\n\nDevelopers hate monthly subscriptions. Prepaid gives them a sense of full control.\n\n**4. Documentation is a feature.**\n\nNo matter how good your architecture is, if the docs are bad, nobody will use it.\n\nIf you work with multiple AI models regularly, give SarangAI a try:\n\nInstall the CLI:\n\n```\nnpm install -g sarangai-cli\n```\n\nI'm curious: **what's your current setup for handling multiple AI providers?** Do you use a library? Or manage them one by one?\n\nShare in the comments - I'd love to hear how other developers handle this.", "url": "https://wpnews.pro/news/how-i-built-a-unified-api-gateway-for-200-ai-models-architecture-deep-dive", "canonical_source": "https://dev.to/sarangai_id/how-i-built-a-unified-api-gateway-for-200-ai-models-architecture-deep-dive-3559", "published_at": "2026-10-06 20:37:16+00:00", "updated_at": "2026-10-06 20:48:29.365430+00:00", "lang": "en", "topics": ["ai-infrastructure", "ai-tools", "large-language-models", "ai-products", "developer-tools"], "entities": ["SarangAI", "OpenAI", "Anthropic", "Google", "Redis", "sarangai-cli"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/how-i-built-a-unified-api-gateway-for-200-ai-models-architecture-deep-dive", "markdown": "https://wpnews.pro/news/how-i-built-a-unified-api-gateway-for-200-ai-models-architecture-deep-dive.md", "text": "https://wpnews.pro/news/how-i-built-a-unified-api-gateway-for-200-ai-models-architecture-deep-dive.txt", "jsonld": "https://wpnews.pro/news/how-i-built-a-unified-api-gateway-for-200-ai-models-architecture-deep-dive.jsonld"}}