{"slug": "your-llm-fallback-probably-isn-t-a-fallback", "title": "Your LLM Fallback Probably Isn't a Fallback", "summary": "A developer's LLM gateway failed when DeepSeek retired two API model names at a V4 cutover, causing all model calls to return HTTP 400. The fallback configuration shared the same provider and API key, so it also failed. The fix was committed within 26 minutes, but the incident revealed a structural flaw in the fallback design.", "body_md": "At 04:00 UTC, every model call through our LLM gateway started returning\n\nHTTP 400. Not some calls. All of them. Our tier-1 CI gate flagged it, and the\n\nfix was committed at 04:26 UTC the same morning — about 26 minutes end to end.\n\nThis is the post-mortem.\n\nDeepSeek retired two API model names — `deepseek-chat`\n\nand `deepseek-reasoner`\n\n—\n\nat their V4 cutover around 2026-07-24 15:59 UTC. The replacements are\n\n`deepseek-v4-pro`\n\nand `deepseek-v4-flash`\n\n.\n\nOur gateway config still declared both retired names. Starting roughly twelve\n\nhours after the retirement, every model request routed through the gateway hit a\n\n400 with the body:\n\nThe supported API model names are deepseek-v4-pro or deepseek-v4-flash, but\n\nyou passed .\n\nA live API check confirmed the shape of the cutover with four requests, same\n\nvalid key:\n\n| Model name | Response |\n|---|---|\n`deepseek-v4-pro` |\nHTTP 200 |\n`deepseek-v4-flash` |\nHTTP 200 |\n`deepseek-chat` |\nHTTP 400 |\n`deepseek-v4-pro-quantized` |\nHTTP 400 |\n\nThe two working names are the replacements. The two retired names — the ones our\n\nconfig referenced — returned 400. The fourth row is a name that does not exist\n\nat all, included because an earlier reading of a truncated error message had\n\nsuggested it; shipping it would have left the platform broken. We'll come back\n\nto that.\n\nWe had a fallback configured. Three separate model references in our policy\n\nconfig — the default CLI/workflow model, the chat model, and the shared fallback\n\nmodel — all pointed at the two retired names. All three lived under the same\n\nvendor and the same API key.\n\nWhen the primary call returned 400, the gateway tried the fallback. The log told\n\nthe story in two adjacent lines: the 400 from the provider, and then\n\n`Error doing the fallback:`\n\ncarrying the identical error. The fallback died in\n\nthe same instant as the primary because it was the same thing wearing a\n\ndifferent label.\n\nThis is the structural problem. A fallback that shares a provider and an API key\n\nwith its primary is not resilience. It protects against a single model\n\nmisbehaving. It protects against nothing at the provider level — an outage, a\n\ncredential revocation, or a model rename all take down primary and fallback\n\ntogether. It also makes the \"does the fallback work?\" test structurally\n\nunpassable during any provider incident, because a red result cannot distinguish\n\n\"fallback broken\" from \"provider down.\"\n\nOur tier-1 CI test gate runs a minimal LLM reachability check on every push. On\n\nthe failing run it reported:\n\n```\nllm_reachability.py: FAIL: P2 'deepseek-reasoner' reachable\nFAIL: P3 pi node executes end-to-end\nFAIL: P4 chat succeeded despite dead primary\n```\n\nThe gate's drift check flagged it as a regression — 108 of 109 checks passing,\n\none new failure. The immediately preceding run of that same gate had been fully\n\ngreen, which proved the cause was external, not something we had just committed.\n\nA gate that catches a provider-side model retirement in a single CI run is the\n\ndifference between finding out from a dashboard and finding out from a user.\n\nTwo lines of gateway config: repoint the two aliases at `deepseek-v4-pro`\n\nand\n\n`deepseek-v4-flash`\n\n. The alias names were left unchanged, so nothing downstream\n\nthat referenced them had to change.\n\nThe mapping was chosen by running the live API check above — not by reading the\n\nerror string. An earlier reading of a truncated error message had suggested a\n\nmodel name (`deepseek-v4-pro-quantized`\n\n) that does not exist. Shipping that\n\nwould have left the platform broken. Four seconds of curl beat a confident\n\nguess.\n\nAfter the fix, both aliases returned HTTP 200 from inside the running pod, and\n\nthe tier-1 gate returned to fully green with no drift.\n\nThe config fix closed the immediate outage. The real defect is the fallback\n\ndesign: same provider, same key, shared fate. We filed it as a tracked gap. It\n\nwill stay open until the fallback crosses a provider boundary and carries\n\nindependent credentials.\n\nHere is a concrete checklist you can run against your own setup right now:\n\n**Does your fallback cross a provider boundary?** If primary and fallback\n\nboth live under the same vendor and the same API key, they share a fate.\n\nA provider outage, credential revocation, or model deprecation will take\n\nboth out simultaneously.\n\n**Do you have a reachability canary?** A single HTTP 200 check per declared\n\nmodel, running in CI on every push (or on a short cron), would have caught\n\nthis in minutes. The check should be cheap — one token of output is enough\n\nto prove the model is alive.\n\n**Would you find out from a gate or from a user?** If your answer is \"a\n\nuser,\" your monitoring has a coverage gap. The test that catches this does\n\nnot need to be in production — a CI gate that fails on a model returning\n\n400 is sufficient and costs almost nothing.\n\nThis incident came from a small platform still under active development. The\n\nlesson generalises to any system that routes LLM requests through a gateway.", "url": "https://wpnews.pro/news/your-llm-fallback-probably-isn-t-a-fallback", "canonical_source": "https://dev.to/gad_ofir_076c468dd15d483b/your-llm-fallback-probably-isnt-a-fallback-34fk", "published_at": "2026-07-25 06:20:22+00:00", "updated_at": "2026-07-25 07:03:20.260895+00:00", "lang": "en", "topics": ["large-language-models", "developer-tools", "ai-infrastructure"], "entities": ["DeepSeek", "LLM gateway"], "alternates": {"html": "https://wpnews.pro/news/your-llm-fallback-probably-isn-t-a-fallback", "markdown": "https://wpnews.pro/news/your-llm-fallback-probably-isn-t-a-fallback.md", "text": "https://wpnews.pro/news/your-llm-fallback-probably-isn-t-a-fallback.txt", "jsonld": "https://wpnews.pro/news/your-llm-fallback-probably-isn-t-a-fallback.jsonld"}}