Calling LLM providers directly works until it doesn't. The moment you're managing multiple API keys, tracking costs across providers, or scrambling during a provider outage, you need a gateway.
But not all AI gateways are created equal. Some are pure proxies. Some lock you into their ecosystem. Some charge you for features that should be free.
We evaluated eight AI gateways on what actually matters: provider coverage, pricing transparency, self-hosting, observability, and how fast you can get started.
Best overall. Open source. Self-hostable.
LLM Gateway is an open-source API gateway that routes requests to 200+ models across 40+ providers through a single OpenAI-compatible endpoint. Change your base URL, keep your existing code.
What sets it apart:
@llmgateway/ai-sdk-provider
Pricing: Free tier with credits to start. Pay-as-you-go with no hidden fees. No markup when using your own keys.
Best for: Teams that want full control over their AI infrastructure without building it from scratch.
import OpenAI from "openai";
const client = new OpenAI({
apiKey: "YOUR_GATEWAY_KEY",
baseURL: "https://api.llmgateway.io/v1",
});
// Works with any of 200+ models
const response = await client.chat.completions.create({
model: "claude-sonnet-4-5",
messages: [{ role: "user", content: "Hello!" }],
});
Large model catalog, community-driven.
OpenRouter aggregates models from many providers and offers a unified API. It has a wide selection of models including open-source and fine-tuned variants.
Strengths:
Weaknesses:
Pricing: Pay-as-you-go with a 5.5% credit-purchase fee; provider token rates pass through with no markup.
Best for: Developers who want quick access to a wide variety of models and don't need self-hosting.
Enterprise-focused with governance features.
Portkey positions itself as an AI gateway for enterprises, with emphasis on observability, guardrails, and compliance features.
Strengths:
Weaknesses:
Pricing: Free tier with limited requests. Paid plans for higher volume and enterprise features.
Best for: Large enterprises with strict compliance and governance requirements.
Open-source proxy with broad provider support.
LiteLLM is an open-source Python proxy that translates OpenAI-compatible requests to 100+ providers. It's popular as a self-hosted solution.
Strengths:
Weaknesses:
Pricing: Free and open source. You pay for your own infrastructure.
Best for: Python teams comfortable with self-hosting and managing their own infrastructure. Outgrowing it? See the best LiteLLM alternatives.
Observability-first with proxy capabilities.
Helicone started as an LLM observability platform and added gateway features. It excels at logging, monitoring, and cost tracking. Note: Helicone was acquired by Mintlify in 2026 and is now in maintenance mode rather than active development.
Strengths:
Weaknesses:
Pricing: Free tier with limited requests. Usage-based pricing beyond that.
Best for: Teams whose primary need is visibility into LLM usage rather than routing and failover.
Zero markup, deep AI SDK integration.
Vercel AI Gateway routes to hundreds of models across 45+ providers through one endpoint, with both OpenAI- and Anthropic-compatible APIs. It went GA in 2025 and is the default provider for the Vercel AI SDK.
Strengths:
@ai-sdk/gateway
)Weaknesses:
Pricing: Pay-as-you-go credits with no token markup. Paid add-ons for custom reporting and team-wide governance.
Best for: Teams building on the Vercel AI SDK who want zero markup and tight ecosystem integration.
Edge-based with Cloudflare ecosystem integration.
Cloudflare AI Gateway leverages their edge network to proxy and cache LLM requests. Tight integration with the Cloudflare ecosystem.
Strengths:
Weaknesses:
Pricing: Free tier included with Cloudflare account. Paid plans for higher limits.
Best for: Teams already invested in the Cloudflare ecosystem who want basic gateway features.
Cloud-native for AWS shops.
AWS Bedrock provides access to foundation models through AWS infrastructure. It's less of a traditional gateway and more of a managed model access layer within AWS.
Strengths:
Weaknesses:
Pricing: Pay-per-token with AWS pricing. No upfront costs.
Best for: Organizations fully committed to AWS that need models within their existing cloud infrastructure.
| Feature | LLM Gateway | OpenRouter | Portkey | LiteLLM | Helicone | Vercel AI GW | Cloudflare AI GW | AWS Bedrock |
|---|---|---|---|---|---|---|---|---|
| Open Source | ||||||||
| Yes | No | Partial | Yes | Yes | No | No | No | |
| Self-Hostable | ||||||||
| Yes | No | Partial | Yes | Yes | No | No | No | |
| BYOK (No Markup) | ||||||||
| Yes | After 1M | Yes | Yes | N/A | Yes | No | N/A | |
| OpenAI-Compatible | ||||||||
| Yes | Yes | Yes | Yes | Yes | Yes | Yes | No | |
| Built-in Caching | ||||||||
| Yes | Beta | Yes | Built-in | No | Yes | Yes | No | |
| Analytics Dashboard | ||||||||
| Yes | Yes | Yes | Built-in | Yes | Yes | Basic | CloudWatch | |
| Automatic Failover | ||||||||
| Yes | Yes | Yes | Config | Limited | Yes | No | No | |
| Guardrails | ||||||||
| Enterprise | Enterprise | Yes | No | No | No | No | Yes | |
| Models | ||||||||
| 300+ | 400+ | 1,600+ | 100+ | N/A | Hundreds | ~20 | ~30 | |
| Playground | ||||||||
| Yes | Yes | No | No | No | Yes | No | Yes |
You want full control and no lock-in: LLM Gateway is the only option that's open source, self-hostable, and charges zero markup on your own keys. You get enterprise features without enterprise pricing.
You want the widest model selection: LLM Gateway and OpenRouter both offer extensive catalogs. LLM Gateway covers 200+ models with the added benefit of self-hosting and BYOK.
You want observability above all: Helicone is purpose-built for logging and analytics, though LLM Gateway and Portkey offer comparable dashboards with full gateway capabilities included.
You're locked into a cloud provider: AWS Bedrock makes sense if your entire stack is AWS and you need models within that boundary. Just know you're trading flexibility for integration.
You're on the Vercel AI SDK: Vercel AI Gateway is the natural fit with zero token markup — though you trade away self-hosting and bring-your-own-infrastructure options.
You're cost-conscious: Gateways that support BYOK (LLM Gateway, LiteLLM, Vercel) let you avoid middleman markup entirely. Over thousands of daily requests, the savings compound fast.
If you want to try the top pick, you can be running in under two minutes:
https://api.llmgateway.io/v1
That's it. Your existing code works. Every request gets logged, cached, and tracked automatically.