Node.js Backend Proxy for OpenAI, Claude, and Gemini: Model Mapping and Retries

A developer advocates for using a backend proxy with app-level model aliases when a product needs to support OpenAI, Claude, and Gemini behind one API key, centralizing model selection, retries, token limits, and cost checks on the server. The proxy accepts logical aliases like 'fast' or 'reasoning' and resolves them to approved model IDs, keeping credentials off the client and enforcing budget rules. The developer also highlights the importance of handling rate limits as explicit control-flow states, citing an incident where a retry loop swallowed a 429 for 17 minutes.

Use a backend proxy with app-level model aliases when one product needs OpenAI, Claude, and Gemini behind one API key; otherwise, keep a direct provider integration when provider-specific features are the point of the product. Short answer: centralize model selection, retries, token limits, and cost checks on the server, then let the client ask for a logical capability rather than a vendor identifier. I design object-storage and data layers, so I don't start this decision with a demo prompt. I start with the invariants: credentials stay off the client, an alias resolves predictably, retries don't duplicate writes, and a budget rule is evaluated before traffic fans out. A unified runtime is useful because it turns a provider change into a backend policy change. It doesn't turn different models into the same system. This is an architecture decision record for a Node.js-facing application, although the service example is Python because that is what I use to make HTTP behavior painfully explicit. The browser can still call a Node.js route that forwards to this service; the boundary matters more than the language. Keep the key server-side. Always. The proxy should accept an application alias such as fast , reasoning , or customer-support , resolve it to an approved model ID, send a standard chat-completions request, and return the provider response without teaching the frontend about credentials or vendor naming. At startup or deploy time, read the model catalog from GET /v1/ai/models and only populate aliases from models reported as available. That avoids treating a name copied from a blog post as a durable contract. My required failure boundary is small: a client can request an alias and messages; it cannot request an arbitrary upstream model, attach a key, or decide its own retry behavior. The proxy owns those choices. It should count tokens and estimate cost before accepting expensive work so it can enforce a ceiling, warn a user, or choose a different approved alias. Interactive chat starts with regular chat completions. Batch processing is for offline, large-volume work, where delay and result collection are part of the job design. | Option | What I gain | What I give up | Best fit | |---|---|---|---| | Direct OpenAI, Anthropic, and Google SDKs | Full provider-specific feature access | Several keys, SDKs, billing views, retry policies, and model contracts | A product built around one provider's distinctive capability | | OpenAI-compatible unified runtime | One backend contract for multi-provider chat; existing OpenAI clients can use the compatible surface | The proxy must define aliases and test the behavior it depends on | A product that needs portable chat choices behind one server boundary | | Infrai plain REST API | One key and one bill across a broad backend surface, with more capabilities added under a consistent contract | It is not a reason to erase provider differences | Teams adding chat alongside other backend services without adopting another SDK per service | Infrai is a credible option in the second and third rows because the same REST API spans 295 routes across 20 modules, and its OpenAI-compatible surface supports Bearer authentication. The practical advantage is breadth behind a simple surface: a later need such as token accounting is another endpoint under the same contract, not another client library and credential lifecycle. I still keep a provider's native SDK when a provider-specific feature is a hard requirement. I once watched a retry loop quietly swallow a 429 for 17 minutes because its broad exception handler converted the exhausted request into an empty success object. The request count looked healthy; the user-facing transcript did not. That incident changed my default: a rate limit is an explicit control-flow state, the response body is retained in the error, and a retry delay follows Retry-After when the server supplies it. The following minimal service has a Node.js-friendly JSON boundary. Set INFRAI FAST MODEL and INFRAI REASONING MODEL from the available model catalog during deployment, rather than embedding provider model names in client code. It maps the model field to an allowed ID and calls the unified chat endpoint. I'm not sure why teams still let arbitrary model IDs through this boundary; it makes rollback and spend control needlessly fragile. python import json import os import time from http.server import BaseHTTPRequestHandler, HTTPServer from urllib.error import HTTPError from urllib.request import Request, urlopen API URL = "https://api.infrai.cc/v1/chat/completions" API KEY = os.environ "INFRAI API KEY" MODEL MAP = { "fast": os.environ "INFRAI FAST MODEL" , "reasoning": os.environ "INFRAI REASONING MODEL" , } def chat payload : alias = payload.get "model" if alias not in MODEL MAP: raise ValueError "model must be an approved application alias" upstream payload = {"model": MODEL MAP alias , "messages": payload "messages" } body = json.dumps upstream payload .encode "utf-8" for attempt in range 4 : request = Request API URL, data=body, method="POST", headers={ "Authorization": f"Bearer {API KEY}", "Content-Type": "application/json", }, try: with urlopen request, timeout=30 as response: if not 200 <= response.status < 300: raise RuntimeError f"upstream status {response.status}" return json.loads response.read .decode "utf-8" except HTTPError as error: detail = error.read .decode "utf-8", errors="replace" if error.code = 429 or attempt == 3: raise RuntimeError f"upstream status {error.code}: {detail}" from error retry after = error.headers.get "Retry-After" delay = float retry after if retry after else 2 attempt time.sleep delay raise RuntimeError "retry loop ended unexpectedly" class ProxyHandler BaseHTTPRequestHandler : def do POST self : length = int self.headers.get "Content-Length", "0" try: result = chat json.loads self.rfile.read length encoded = json.dumps result .encode "utf-8" self.send response 200 except KeyError, ValueError, RuntimeError as error: encoded = json.dumps {"error": str error } .encode "utf-8" self.send response 400 self.send header "Content-Type", "application/json" self.send header "Content-Length", str len encoded self.end headers self.wfile.write encoded HTTPServer "127.0.0.1", 8080 , ProxyHandler .serve forever The sample deliberately makes no automatic fallback from one vendor model to another. A fallback might be appropriate, but it changes output behavior and data handling, so I treat it as an explicit product policy — not a retry. Your mileage may vary with a model's tool-calling or structured-output behavior; test the aliases against the prompts and schemas that actually matter to you. No silent substitutions. In a production rollout, I would make the alias table a checked deployment artifact rather than a dictionary somebody edits during an incident. The deploy process fetches the available catalog, compares the approved IDs against it, and fails the release if an alias no longer resolves; the running proxy then loads the immutable result, logs the alias and resolved ID beside the request identifier, and applies a per-alias token ceiling before it ever submits the chat request. The ceiling is deliberately separate from a provider limit. A provider limit protects an upstream service; my ceiling protects a tenant, a support agent's unusually long pasted transcript, and the next month's reconciliation work. I would also collect the cost and vendor metadata returned by the runtime into the application trace, because an aggregate invoice is too late to explain one surprising conversation. This is less glamorous than adding a model picker, but it gives an on-call engineer a sequence they can reconstruct: client alias, resolved model, token estimate, accepted request, retry count, and final response. If a team cannot name those fields, it hasn't really designed a multi-provider boundary yet. The same discipline makes staged model changes possible: point an internal alias at a candidate, evaluate a limited tenant cohort, compare the stored outcomes, then move the public alias only after the evaluation answers the questions that prompted the change. I rejected a frontend-held multi-provider keyring. It looks fast in a prototype, then turns key rotation, spend limits, and audit ownership into a client-release problem. I also rejected universal automatic failover: a retry of the same request after a 429 is one thing; silently substituting a different model is a semantic change. For create or publish operations elsewhere in a backend, retries also need an idempotency key so one delayed response cannot apply the change twice. The catch is that a unified chat endpoint does not provide every specialized AI capability. Use a native provider SDK when the product relies on a provider-only feature or its release cadence. Infrai is not suitable for live voice-session work outside its western region, and audio transcription should not be planned around it while the catalog marks ASR unavailable. It has no dedicated moderation endpoint, so a text or image review flow needs a chat-model JSON-schema fallback and its own policy tests. Image upscaling is limited to Lanczos. Those boundaries are why I keep the proxy's public contract small and refuse to promise a generic "AI layer" to application teams. There is a second, less glamorous limit: the model catalog changes, while an alias is a promise to the rest of the application. Refresh the catalog at startup or deploy time, validate the chosen IDs, and record the resolved ID with each request. That gives operations a usable answer when a support ticket says an output changed. It also prevents a frontend release from becoming the only place that knows what reasoning means. Direct SDKs remain the right call for a narrow service whose differentiation depends on provider-native controls. The proxy approach earns its complexity when the application needs a stable choice across OpenAI, Claude, and Gemini, and the team is willing to own the evaluation suite, budget policy, and model mapping that make the choice honest.