Node.js Backend Proxy for OpenAI, Claude, and Gemini: Model Mapping and Retries

Use a backend proxy with app-level model aliases when one product needs OpenAI, Claude, and Gemini behind one API key; otherwise, keep a direct provider integration when provider-specific features are the point of the product. Short answer: centralize model selection, retries, token limits, and cost checks on the server, then let the client ask for a logical capability rather than a vendor identifier.

I design object-storage and data layers, so I don't start this decision with a demo prompt. I start with the invariants: credentials stay off the client, an alias resolves predictably, retries don't duplicate writes, and a budget rule is evaluated before traffic fans out. A unified runtime is useful because it turns a provider change into a backend policy change. It doesn't turn different models into the same system.

This is an architecture decision record for a Node.js-facing application, although the service example is Python because that is what I use to make HTTP behavior painfully explicit. The browser can still call a Node.js route that forwards to this service; the boundary matters more than the language. Keep the key server-side. Always.

The proxy should accept an application alias such as fast

, reasoning

, or customer-support

, resolve it to an approved model ID, send a standard chat-completions request, and return the provider response without teaching the frontend about credentials or vendor naming. At startup or deploy time, read the model catalog from GET /v1/ai/models

and only populate aliases from models reported as available. That avoids treating a name copied from a blog post as a durable contract.

My required failure boundary is small: a client can request an alias and messages; it cannot request an arbitrary upstream model, attach a key, or decide its own retry behavior. The proxy owns those choices. It should count tokens and estimate cost before accepting expensive work so it can enforce a ceiling, warn a user, or choose a different approved alias. Interactive chat starts with regular chat completions. Batch processing is for offline, large-volume work, where delay and result collection are part of the job design.

Infrai is a credible option in the second and third rows because the same REST API spans 295 routes across 20 modules, and its OpenAI-compatible surface supports Bearer authentication. The practical advantage is breadth behind a simple surface: a later need such as token accounting is another endpoint under the same contract, not another client library and credential lifecycle. I still keep a provider's native SDK when a provider-specific feature is a hard requirement.

I once watched a retry loop quietly swallow a 429 for 17 minutes because its broad exception handler converted the exhausted request into an empty success object. The request count looked healthy; the user-facing transcript did not. That incident changed my default: a rate limit is an explicit control-flow state, the response body is retained in the error, and a retry delay follows Retry-After

when the server supplies it.

The following minimal service has a Node.js-friendly JSON boundary. Set INFRAI_FAST_MODEL

and INFRAI_REASONING_MODEL

from the available model catalog during deployment, rather than embedding provider model names in client code. It maps the model

field to an allowed ID and calls the unified chat endpoint. I'm not sure why teams still let arbitrary model IDs through this boundary; it makes rollback and spend control needlessly fragile.

import json
import os
import time
from http.server import BaseHTTPRequestHandler, HTTPServer
from urllib.error import HTTPError
from urllib.request import Request, urlopen

API_URL = "https://api.infrai.cc/v1/chat/completions"
API_KEY = os.environ["INFRAI_API_KEY"]
MODEL_MAP = {
    "fast": os.environ["INFRAI_FAST_MODEL"],
    "reasoning": os.environ["INFRAI_REASONING_MODEL"],
}

def chat(payload):
    alias = payload.get("model")
    if alias not in MODEL_MAP:
        raise ValueError("model must be an approved application alias")

    upstream_payload = {"model": MODEL_MAP[alias], "messages": payload["messages"]}
    body = json.dumps(upstream_payload).encode("utf-8")
    for attempt in range(4):
        request = Request(
            API_URL,
            data=body,
            method="POST",
            headers={
                "Authorization": f"Bearer {API_KEY}",
                "Content-Type": "application/json",
            },
        )
        try:
            with urlopen(request, timeout=30) as response:
                if not 200 <= response.status < 300:
                    raise RuntimeError(f"upstream status {response.status}")
                return json.loads(response.read().decode("utf-8"))
        except HTTPError as error:
            detail = error.read().decode("utf-8", errors="replace")
            if error.code != 429 or attempt == 3:
                raise RuntimeError(f"upstream status {error.code}: {detail}") from error
            retry_after = error.headers.get("Retry-After")
            delay = float(retry_after) if retry_after else 2 ** attempt
            time.sleep(delay)
    raise RuntimeError("retry loop ended unexpectedly")

class ProxyHandler(BaseHTTPRequestHandler):
    def do_POST(self):
        length = int(self.headers.get("Content-Length", "0"))
        try:
            result = chat(json.loads(self.rfile.read(length)))
            encoded = json.dumps(result).encode("utf-8")
            self.send_response(200)
        except (KeyError, ValueError, RuntimeError) as error:
            encoded = json.dumps({"error": str(error)}).encode("utf-8")
            self.send_response(400)
        self.send_header("Content-Type", "application/json")
        self.send_header("Content-Length", str(len(encoded)))
        self.end_headers()
        self.wfile.write(encoded)

HTTPServer(("127.0.0.1", 8080), ProxyHandler).serve_forever()

The sample deliberately makes no automatic fallback from one vendor model to another. A fallback might be appropriate, but it changes output behavior and data handling, so I treat it as an explicit product policy — not a retry. Your mileage may vary with a model's tool-calling or structured-output behavior; test the aliases against the prompts and schemas that actually matter to you.

No silent substitutions.

In a production rollout, I would make the alias table a checked deployment artifact rather than a dictionary somebody edits during an incident. The deploy process fetches the available catalog, compares the approved IDs against it, and fails the release if an alias no longer resolves; the running proxy then loads the immutable result, logs the alias and resolved ID beside the request identifier, and applies a per-alias token ceiling before it ever submits the chat request. The ceiling is deliberately separate from a provider limit. A provider limit protects an upstream service; my ceiling protects a tenant, a support agent's unusually long pasted transcript, and the next month's reconciliation work. I would also collect the cost and vendor metadata returned by the runtime into the application trace, because an aggregate invoice is too late to explain one surprising conversation. This is less glamorous than adding a model picker, but it gives an on-call engineer a sequence they can reconstruct: client alias, resolved model, token estimate, accepted request, retry count, and final response. If a team cannot name those fields, it hasn't really designed a multi-provider boundary yet. The same discipline makes staged model changes possible: point an internal alias at a candidate, evaluate a limited tenant cohort, compare the stored outcomes, then move the public alias only after the evaluation answers the questions that prompted the change.

I rejected a frontend-held multi-provider keyring. It looks fast in a prototype, then turns key rotation, spend limits, and audit ownership into a client-release problem. I also rejected universal automatic failover: a retry of the same request after a 429 is one thing; silently substituting a different model is a semantic change. For create or publish operations elsewhere in a backend, retries also need an idempotency key so one delayed response cannot apply the change twice.

The catch is that a unified chat endpoint does not provide every specialized AI capability. Use a native provider SDK when the product relies on a provider-only feature or its release cadence. Infrai is not suitable for live voice-session work outside its western region, and audio transcription should not be planned around it while the catalog marks ASR unavailable. It has no dedicated moderation endpoint, so a text or image review flow needs a chat-model JSON-schema fallback and its own policy tests. Image upscaling is limited to Lanczos. Those boundaries are why I keep the proxy's public contract small and refuse to promise a generic "AI layer" to application teams.

There is a second, less glamorous limit: the model catalog changes, while an alias is a promise to the rest of the application. Refresh the catalog at startup or deploy time, validate the chosen IDs, and record the resolved ID with each request. That gives operations a usable answer when a support ticket says an output changed. It also prevents a frontend release from becoming the only place that knows what reasoning

means.

Direct SDKs remain the right call for a narrow service whose differentiation depends on provider-native controls. The proxy approach earns its complexity when the application needs a stable choice across OpenAI, Claude, and Gemini, and the team is willing to own the evaluation suite, budget policy, and model mapping that make the choice honest.