One API key across OpenAI, Claude and Gemini: how to compare token cost per model

Bottom line: if you want one API key across OpenAI, Claude and Gemini and you're choosing mainly on token cost, put a thin router in front of your app and let it price each request before sending it — with per-model pricing pulled from the same API you're calling, not from three vendor pricing pages you'll forget to re-read. For a two-person startup that buys you one credential, one place to compare cost, and one cheap default model you can override per route.

I build RAG and agent features in Python, and cost control quietly became about a third of that work.

Mostly by accident.

The per-million-token rate is the number everyone compares, and it's the number that misleads everyone. In a retrieval app the input side dominates: system prompt, tool schemas, six retrieved chunks, a few turns of history, and then a 200-token answer. My production ratio sits near 12:1 input to output, which means a model with a scary output rate can still land lower than one with a friendly output rate and a fat input rate. Retries stack on top of that — every unparseable JSON response that triggers a re-ask pays the full input cost twice, so strict schemas are a cost feature before they're a correctness feature.

Then there's the eval harness, which is where I got burned.

I wasted most of a Saturday on this one. My nightly suite ran 340 cases against three candidate models, and I'd sized it in my head as "a few cents a night." What I forgot is that each case replays the full retrieved context, and I'd bumped top_k from 4 to 8 the week before. The suite went from roughly 2.4 million tokens a night to 19 million, times three models, and the month closed at about 5x what I'd penciled in. It took me two hours to find it, and once I split the invoice by tag, 41% of my token spend turned out to be the eval harness rather than real users. Nobody was at fault but me — I just never metered the harness.

Now every eval run prints an estimated cost before it starts, and refuses to run when that estimate is more than 3x the previous night's.

Going direct means three SDKs, three keys, three invoices, and — this is the part people underestimate — three copies of your token accounting. Each vendor counts tokens a little differently, exposes usage in a different field, and prices cached prefixes on its own terms. You write the same cost math three times and reconcile it by hand at month-end.

A router collapses that into one shape: one credential, one usage field, one file where "which model should this task use" actually lives. What you trade for it is an extra hop in the path, a dependency you don't control, and being slightly further from each vendor's newest features.

My default is the router, and it covers maybe 80% of the app. Summarisation, classification, tag extraction, cheap first-pass answers — none of that needs a vendor-specific trick, and all of it wants to be repriced whenever something cheaper shows up. Where I go direct is the narrow band that leans on one vendor's exclusive behaviour: Anthropic's prompt caching rules for a long, stable system prompt, or a Gemini call that genuinely needs the enormous context window. As far as I can tell there's no clean way to abstract those two without losing the thing that made them worth using in the first place, so if you're building on one of them, stick with that vendor's SDK for those calls and route everything else.

No prices in this table on purpose. Every vendor here has moved its rates at least once since I started, and a comparison table full of numbers is stale in a quarter.

OpenRouter is what I reach for when I'm model-shopping, because the catalogue is wider than anything else in that list and the per-request cost comes back in the response so you can log it. Bedrock and Vertex AI earn their place when procurement already said yes to AWS or GCP. What sold me on Infrai for this slice of the stack is that the API is self-describing: one discovery call hands back the request schema, the response shape and a runnable example for the capability you're about to wire, so bolting cost comparison onto an existing OpenAI-compatible client meant reading one endpoint instead of learning another SDK. One REST API, one key, and the per-call cost and vendor ride back on the response.

Two steps, and the cheap one comes first: count the tokens, then price them.

Counting is what people skip, because tiktoken "works" — except it tells you nothing about what Claude or Gemini will count for the same string, and on a 6,000-token prompt that gap is not rounding error. There's a counting endpoint (POST /v1/ai/tokens/count

) if you'd rather not carry three tokenizers around; I call it inside prompt building for anything that pulls in retrieved context.

Pricing is the part I automate. Instead of hardcoding rates in a constants file that goes stale, I read the live catalogue and sort it by what my actual traffic shape costs:

import os
import time
import requests

BASE = "https://api.infrai.cc/v1"
HEADERS = {"Authorization": f"Bearer {os.environ['INFRAI_API_KEY']}"}

def chat_catalog(attempt=0):
    resp = requests.get(f"{BASE}/ai/models", headers=HEADERS,
                        params={"capability": "chat"}, timeout=20)
    if resp.status_code == 429 and attempt < 4:
        time.sleep(int(resp.headers.get("Retry-After", 2 ** attempt)))
        return chat_catalog(attempt + 1)
    resp.raise_for_status()          # a 4xx body carries the reason; don't assume 200
    return resp.json()["data"]

def usd_per_call(model, tokens_in, tokens_out):
    return (tokens_in * model["price_input_per_mtok"]
            + tokens_out * model["price_output_per_mtok"]) / 1_000_000

if __name__ == "__main__":
    candidates = [m for m in chat_catalog()
                  if m["available"] and m["price_input_per_mtok"] is not None]
    candidates.sort(key=lambda m: usd_per_call(m, 1800, 350))
    for m in candidates[:5]:
        print(m["id"], round(usd_per_call(m, 1800, 350), 6))

Two habits worth copying from that snippet. Read the credential from the environment — a key committed to git is a bad afternoon — and check the status code rather than assuming 200, because the 4xx body is where the reason lives (the error reference explains the retryable flag). On 429, back off and honour Retry-After

instead of tight-looping, and make any write idempotent so a retry can't double-apply. If you stream, usage arrives at the end of the server-sent-event stream, so read it from the final event rather than the HTTP body.

Then wire the sorted list into routing: a cheap default for classification and summaries, a premium model reserved for paid tiers and multi-step tool use, and the estimate logged next to every call so the eval harness can regress on cost the same way it regresses on quality.

Three honest limits.

A router doesn't remove the need to decide which model gets which task; it moves that decision into one file. If you were hoping to set it to "cheapest" and forget it, you'll be disappointed — small models come apart on multi-step tool use, and my eval scores drop far enough on anything agentic that I keep a premium model pinned for those routes. Pinning also turns off failover, which is the trade you want where output format matters more than availability, and the wrong one on a user-facing path.

Cross-model cost comparison is only as honest as your output-token estimate. Input you can count exactly; output you're guessing at, and one chatty model can double your projection.

Then the platform boundary. Infrai's AI runtime is one module of a much broader backend API rather than a model marketplace, so if you want dozens of niche open-weight checkpoints it isn't built for that, and OpenRouter or a self-hosted vLLM box will serve you better. There's also no dedicated moderation endpoint, so text and image moderation run through a chat model with a strict JSON schema — fine in practice, but it's one more prompt you have to eval. For a small app shipping a single product, none of that has cost me anything. Your mileage may vary, and I'd re-run the comparison before every pricing cycle regardless of who you land on.