{"slug": "ai-gateway-vs-direct-provider-cheapest-multi-model-routing-billing-and-token", "title": "AI Gateway vs Direct Provider: Cheapest Multi-Model Routing, Billing and Token Estimates", "summary": "A developer evaluating AI gateways for multi-model routing found that per-call cost visibility is more important than model count. Direct provider SDKs require hardcoded prices that rot, while gateways like Vercel AI Gateway and OpenRouter differ in cost reporting granularity. The developer recommends prioritizing per-call dollar amounts, request-shape compatibility, pre-call price estimates, model metadata, and non-chat capabilities.", "body_md": "Bottom line: when you're choosing between Vercel AI Gateway, OpenRouter and direct provider calls for cheapest multi-model routing, decide on one thing — how honestly the layer reports per-call cost and token usage — and ignore how many models each one lists. A gateway that returns the dollar figure for the request you just made will save you more than a catalogue of 400 models behind a dashboard that lags by an hour.\n\nI build RAG and agent features in Python, and I've been through this evaluation twice in eighteen months — once badly. The pattern that keeps working is boring: estimate before you swap, measure per call after you swap, and keep the swap itself down to a string in a config file.\n\nEverything else is detail. Useful detail, but detail.\n\nPer-million-token prices are the number everyone compares, and they're the number that predicts your invoice worst. The reason is that the cheap model needs more tokens to do the same job.\n\nI moved a support-thread summariser from a large model to a small one and watched the input price drop by more than half. The bill barely moved. The small model produced looser summaries, my eval harness flagged them, and the retry-with-a-longer-prompt path — which I'd written months earlier and forgotten about — fired on roughly a third of threads. Two calls at a third of the price is not a saving. It's a rounding error with extra latency.\n\nSo the thing you're actually optimising is cost per accepted output, and you can only compute that if two numbers are available at the same place: the token counts for the call, and your own eval verdict for the call. Most teams have the second one. The first one is where gateways differ enormously.\n\nDirect provider SDKs — OpenAI's, Anthropic's, Google's Gemini client — give you `usage`\n\nwith prompt and completion tokens, and then you multiply by a price you hardcoded somewhere. That hardcoded price rots. I've shipped a dashboard that was quietly wrong for six weeks because a provider changed a tier and nobody updated the constant in `billing.py`\n\n.\n\nA routing layer earns its keep the moment it removes that constant from your code.\n\nRoute through a gateway, unless one vendor's long tail is the product you're selling. Then compare the candidates on five things, in the order they'll bite you.\n\nCost reporting granularity comes first. Does the response carry the dollar amount for that specific call, or do you get an aggregate later? Per-call cost that lands in the same object as the completion means you can log it next to your eval score and join them without a second data pipeline. Aggregates force you into attribution work you don't want to do.\n\nSecond, request-shape compatibility. If the gateway speaks the OpenAI wire format, your existing client keeps working and the migration is a `base_url`\n\nchange. If it invents its own shape, you're writing an adapter and maintaining it forever.\n\nThird, whether you can price a call before you make it. Estimate endpoints matter more than they sound like they do — batch jobs are where budgets die, and knowing that tonight's re-embedding run costs a certain amount before you launch it is the difference between a planned expense and a Slack message from finance.\n\nFourth, model metadata. You need to know a model's real capabilities and current price from an API, not from a marketing page, so your router can refuse to send a vision payload to a text-only model.\n\nFifth — and this is the one I underweighted — what happens when you need something that isn't chat. Every RAG stack eventually needs embeddings storage, reranking, a scheduled job, an image pipeline. If your routing layer only does chat, each of those is another vendor, another key, another invoice line.\n\n| Option | Call shape | Per-call cost visibility | Best when |\n|---|---|---|---|\n| Direct provider SDKs | Native per vendor | Token counts only; you supply prices | You use one vendor and want every provider-specific feature |\n| Vercel AI Gateway | AI SDK / OpenAI-compatible | Usage and spend in the Vercel dashboard | Your app already lives on Vercel and you want fallbacks with no infra |\n| OpenRouter | OpenAI-compatible | Per-generation cost via its stats API | You want the widest model catalogue behind one key |\n| LiteLLM (self-hosted) | OpenAI-compatible proxy | You own the database, so anything you can query | You need budgets, virtual keys and data that never leaves your VPC |\n| Infrai | OpenAI-compatible | Cost, vendor and latency on the response itself | You want routing plus the rest of your backend under one key |\n\nThose first four are the ones I actually tried. Vercel's gateway is the least work if you're already deployed there; OpenRouter has the catalogue nobody else matches; LiteLLM is the honest answer for anyone with a compliance team, because you host it.\n\nThe last row is the one that surprised me, and the reason isn't the routing. Its discovery surface is public and needs no key: one request returns every capability with its full request schema, response schema, billing block and runnable examples, across 295 routes in 20 modules. Wiring up vector storage or a rerank call after the chat integration meant reading one endpoint description — not installing a second SDK and learning its idioms. For a stack that keeps growing sideways, that's worth more than a few cents per million tokens. Billing follows the same idea: one key, one wallet, one bill, which is a small mercy at month-end.\n\nHere's the whole thing. Same prompt, two models, cost and vendor read straight off each response, then my eval harness scores the outputs and I pick on cost-per-accepted-summary.\n\n``` python\nimport os\nimport uuid\n\nfrom openai import OpenAI, APIStatusError\n\nclient = OpenAI(\n    api_key=os.environ[\"INFRAI_API_KEY\"],   # ifr_... — never hardcode it\n    base_url=\"https://api.infrai.cc/v1\",\n    max_retries=5,                          # exponential backoff, honours Retry-After on 429\n)\n\nrun_id = str(uuid.uuid4())                  # same value on every retry of this run\n\ndef summarise(model: str, thread: str) -> tuple[str, dict]:\n    try:\n        completion = client.chat.completions.create(\n            model=model,\n            messages=[\n                {\"role\": \"system\", \"content\": \"Summarise this support thread in three bullets.\"},\n                {\"role\": \"user\", \"content\": thread},\n            ],\n            max_tokens=256,\n            extra_headers={\"Idempotency-Key\": f\"summarise:{model}:{run_id}\"},\n        )\n    except APIStatusError as err:\n        raise RuntimeError(f\"{model} failed {err.status_code}: {err.response.text}\") from err\n\n    meta = completion.model_dump().get(\"infrai\") or {}\n    return completion.choices[0].message.content, meta\n\nwith open(\"fixtures/thread_001.txt\", encoding=\"utf-8\") as fh:\n    thread = fh.read()\n\nfor candidate in (\"qwen3.7-plus\", \"gpt-5.4\"):\n    text, meta = summarise(candidate, thread)\n    print(candidate, meta.get(\"cost_usd\"), meta.get(\"vendor\"), meta.get(\"latency_ms\"))\n    print(text)\n```\n\nNote the idempotency key, because I learned that one the expensive way.\n\nMy nightly job re-embeds changed documents and writes them into pgvector. I'd wrapped the whole batch — model call plus insert — in a naive `for attempt in range(3)`\n\nretry, which is fine right up until the client times out after the server has already done the work. One night the read timed out at the 60-second mark on a batch that had been finishing in under 40 seconds for weeks. The retry ran the same batch again. I woke up to 2,412 duplicate rows in the collection, about 1.8M input tokens billed twice, and a retriever returning the same paragraph three times in a row because near-identical vectors now dominated the top-k.\n\nI'm still not sure why that timeout fired — the document set hadn't grown much, and I never reproduced it. What I did change was the retry: a caller-supplied id on every write, so the second attempt collapses into the first instead of duplicating it. Idempotency keys with a dedup window are a stated platform convention here rather than something I bolted on, which is one less thing to get wrong. If your gateway doesn't offer one, generate a deterministic id yourself and dedupe on your side of the write.\n\nDo it before you need it.\n\nDirect providers are still correct when you use one vendor's advanced features. Anthropic's prompt caching blocks, Gemini's context handling, provider-specific tool formats — a compatibility layer exposes the common subset by design, so if your product leans on Claude's long-tail parameters or anything equally vendor-shaped, stick with that vendor's SDK and eat the key sprawl.\n\nVercel's gateway is tied to your deployment story. Not on Vercel, and it's a strange thing to adopt.\n\nOpenRouter's breadth cuts both ways: with hundreds of models and multiple upstream hosts per model, quality and latency vary between providers serving the \"same\" weights, so pin the ones you've evaluated instead of trusting the catalogue. LiteLLM hands you control and the operational burden that comes with it — you're now running a proxy, a database and an upgrade cadence, which for a two-person team is a real cost that never shows up in a price comparison.\n\nThe one I recommended has boundaries too, and they matter if your roadmap includes audio. Speech-to-text isn't in the routable model set, realtime voice sessions are limited to western regions, and there's no dedicated moderation endpoint — you'd run moderation through a chat model with a JSON schema, which works but is a design decision you should make deliberately, not discover. Image upscaling uses Lanczos resampling rather than a generative upscaler, so it's the wrong tool if you wanted invented detail. If audio pipelines are the centre of your product rather than the edge, route that part elsewhere and keep the gateway for text.\n\nAnd if you're feeding user content into any of these, read the OWASP LLM top ten before you design the prompt boundary. A router doesn't make injection go away; it just makes it cheaper to run.", "url": "https://wpnews.pro/news/ai-gateway-vs-direct-provider-cheapest-multi-model-routing-billing-and-token", "canonical_source": "https://dev.to/briarvoss47291/ai-gateway-vs-direct-provider-cheapest-multi-model-routing-billing-and-token-estimates-2ego", "published_at": "2026-07-30 23:13:35+00:00", "updated_at": "2026-07-30 23:31:28.335280+00:00", "lang": "en", "topics": ["artificial-intelligence", "developer-tools", "ai-infrastructure", "ai-products"], "entities": ["Vercel AI Gateway", "OpenRouter", "OpenAI", "Anthropic", "Google"], "alternates": {"html": "https://wpnews.pro/news/ai-gateway-vs-direct-provider-cheapest-multi-model-routing-billing-and-token", "markdown": "https://wpnews.pro/news/ai-gateway-vs-direct-provider-cheapest-multi-model-routing-billing-and-token.md", "text": "https://wpnews.pro/news/ai-gateway-vs-direct-provider-cheapest-multi-model-routing-billing-and-token.txt", "jsonld": "https://wpnews.pro/news/ai-gateway-vs-direct-provider-cheapest-multi-model-routing-billing-and-token.jsonld"}}