Vercel AI Gateway Virtual Models: Swap Providers Safely A developer published a guardrail guide for Vercel's newly shipped virtual model IDs on AI Gateway, which let teams swap the provider, fallback chain and underlying model behind a request centrally without deploying application code. The post recommends keeping the model ID in a single source of truth, routing all calls through one wrapper that logs both the requested and response model IDs, and codifying a fallback policy that retries on timeouts, 429s and 5xx errors but fails fast on 400s and policy refusals. Vercel just shipped virtual model IDs on AI Gateway, so you can change the provider, the fallback chain and the model behind a request centrally, without deploying a single line of application code. That's a great escape hatch. It's also a new way to change production behavior with no pull request, no diff and no test run. This post is the guardrail kit I'd wire up before the first person flips a route in the dashboard. TL;DR Here's the change in one sentence: your code asks for a stable name, and the gateway decides what that name means today. If you've read my guide to LLM model routing for cost and quality https://mudassirkhan.me/blog/llm-model-routing , this is the same idea moved out of your codebase and into the platform. The first rule is boring and it matters most: the model ID lives in exactly one place. Not in five route handlers, not in a prompt file, not in a feature flag someone forgot about. // src/ai/models.ts // One source of truth for which model name the app asks for. // AI MODEL ID holds the virtual model ID you created in the AI Gateway dashboard. // AI MODEL PINNED holds a concrete provider/model ID you trust, used for evals and rollback. function requireEnv name: string : string { const value = process.env name ; if value throw new Error Missing env var ${name} ; return value; } export const MODELS = { chat: requireEnv "AI MODEL ID" , pinned: requireEnv "AI MODEL PINNED" , } as const; export type ModelRole = keyof typeof MODELS; Then every call goes through one function: js // src/ai/generate.ts import { generateText } from "ai"; import { MODELS, type ModelRole } from "./models"; export async function ask prompt: string, role: ModelRole = "chat" { const requested = MODELS role ; const result = await generateText { model: requested, prompt } ; // Log both sides: what you asked for and what the response says answered. console.log JSON.stringify { requested model: requested, response model: result.response.modelId, input tokens: result.usage.inputTokens, output tokens: result.usage.outputTokens, } ; return result.text; } The AI SDK sends a plain string model ID through AI Gateway when AI GATEWAY API KEY is set. Check in your own logs what response.modelId reports for a virtual ID before you build dashboards on it. I'd rather you see it with your own eyes than trust my guess about it. Fallbacks sound like free reliability. They aren't. A fallback is a second model answering a question you tuned for the first one, so you need a contract that says when that's allowed. Cloudflare AI Gateway documents the same pattern from the other side: a multistep fallback chain that moves to the next model after a request fails or times out, plus a cf-aig-step response header that tells you which step answered step 0 is the primary, then 1, then 2 . Whichever gateway you run, that's the shape to copy: a fallback happens for a reason you named, and the response tells you it happened. My rule of thumb: | Error | Fall back? | Why | |---|---|---| | Timeout | Yes | The provider is slow, your prompt is fine | | 429 rate limit | Yes | Capacity problem, not a content problem | | 5xx from the provider | Yes | Their outage, not your bug | | 400 bad request | No, fail fast | A different model won't fix a malformed request | | Policy refusal | No, fail fast and log | Silently retrying elsewhere hides a real decision | If part of your stack still does its own retries a second gateway, an older code path , make that rule explicit in code so the two layers don't fight: // src/ai/fallback-policy.ts export type FailureKind = "timeout" | "rate limit" | "provider down" | "bad request" | "refused"; export function classify status: number | undefined, err: unknown : FailureKind { if err instanceof DOMException && err.name === "TimeoutError" return "timeout"; if status === 429 return "rate limit"; if status == undefined && status = 500 return "provider down"; if status === 400 || status === 422 return "bad request"; return "refused"; } const MAY FALL BACK: ReadonlySet