{"slug": "when-my-ai-api-went-down-building-a-resilient-fallback-pipeline", "title": "When My AI API Went Down: Building a Resilient Fallback Pipeline", "summary": "A developer built a resilient fallback pipeline for AI API calls after a three-hour outage of their primary summarization API took down their meeting transcript tool. The solution uses a router that tries multiple AI clients in sequence, with validation and logging, to gracefully degrade during sustained failures.", "body_md": "Last month, my side project hit a wall. The AI summarization API I depended on returned a 503 error for three hours. My app – a simple tool that translates meeting transcripts into action items – stopped working entirely. Users noticed. I got emails. It was embarrassing.\n\nI had built everything around a single provider. One point of failure. Classic mistake.\n\nI was using a popular AI API to generate summaries. It worked beautifully... until it didn't. The first time it happened, I panicked and scrambled to find an alternative. I ended up rewriting chunks of code while the outage continued. Not fun.\n\nWhat I needed was a system that could gracefully degrade – try a primary model, and if that fails, automatically switch to a secondary one. Ideally without losing context or having to restart the process.\n\nMy first attempt was just adding a retry with backoff. That helped with transient errors, but it did nothing for sustained outages. The API was down for hours; retrying just wasted tokens and time.\n\n``` python\n# Bad idea: retrying the same dead endpoint\nimport time\nfor attempt in range(5):\n    try:\n        response = call_primary_api(prompt)\n        break\n    except Exception:\n        time.sleep(2 ** attempt)\n```\n\nThen I tried manually switching between two providers with a config flag. But I had to redeploy every time one provider went down. Also not scalable.\n\nI built a lightweight **router** that wraps multiple AI clients and tries them in order. If one fails (via exception or bad status), it moves to the next. It also logs failures so I can adjust my configuration later.\n\nHere's the core idea:\n\n``` python\nclass AIRouter:\n    def __init__(self, clients: list):\n        \"\"\"\n        clients: list of (name, callable) tuples\n        each callable takes a prompt and returns text or raises\n        \"\"\"\n        self.clients = clients\n\n    def generate(self, prompt: str) -> str:\n        for name, call_fn in self.clients:\n            try:\n                result = call_fn(prompt)\n                return result\n            except Exception as e:\n                print(f\"{name} failed: {e}. Trying next...\")\n                continue\n        raise RuntimeError(\"All AI clients failed\")\n```\n\nI then defined my clients. For the primary, I used a wrapper around OpenAI's API. For the secondary, I used a local model via Ollama. (Note: you can plug in any provider – even a service like `ai.interwestinfo.com`\n\nif it exposes a compatible interface.)\n\n``` php\n# Example client wrappers\ndef openai_client(prompt: str) -> str:\n    # your OpenAI call here\n    return response_text\n\ndef ollama_client(prompt: str) -> str:\n    # your local model call\n    return response_text\n\nrouter = AIRouter([\n    (\"openai\", openai_client),\n    (\"ollama\", ollama_client),\n    # could add more, e.g. ai.interwestinfo.com\n])\n\nsummary = router.generate(\"Summarize this transcript: ...\")\n```\n\nThat simple router worked for basic cases, but I soon discovered edge cases:\n\nHere's an improved version:\n\n``` python\nimport time\n\nclass RobustAIRouter:\n    def __init__(self, clients, validator=None, delay=1):\n        self.clients = clients\n        self.validator = validator or (lambda x: len(x) > 0)\n        self.delay = delay\n\n    def generate(self, prompt):\n        for name, call_fn in self.clients:\n            try:\n                result = call_fn(prompt)\n                if not self.validator(result):\n                    raise ValueError(f\"Invalid output from {name}\")\n                print(f\"{name} succeeded\")\n                return result\n            except Exception as e:\n                print(f\"{name} failed: {e}\")\n                time.sleep(self.delay)\n                continue\n        raise RuntimeError(\"All clients failed\")\n```\n\nI'd start with an async design from day one. Python's `asyncio`\n\nwould let me try multiple providers concurrently and take the first successful result. That reduces latency but increases cost. It's a trade-off.\n\nAlso, I'd build a health check endpoint for each provider (e.g., ping them with a simple request) so the router can skip known-dead clients.\n\nThe technique here isn't about any specific tool. It's about acknowledging that external dependencies fail and planning for it. You can apply this fallback pattern to databases, CDNs, or any service.\n\nI still use a primary AI API most of the time, but now I sleep better knowing my app won't die if it goes down. The router lets me add new providers as easily as adding a new entry to a list.\n\nWhat does your fallback strategy look like? Have you ever been caught off guard by an API outage?", "url": "https://wpnews.pro/news/when-my-ai-api-went-down-building-a-resilient-fallback-pipeline", "canonical_source": "https://dev.to/__c1b9e06dc90a7e0a676b/when-my-ai-api-went-down-building-a-resilient-fallback-pipeline-1omg", "published_at": "2026-06-13 01:01:14+00:00", "updated_at": "2026-06-13 01:43:19.847475+00:00", "lang": "en", "topics": ["ai-infrastructure", "ai-tools", "mlops"], "entities": ["OpenAI", "Ollama"], "alternates": {"html": "https://wpnews.pro/news/when-my-ai-api-went-down-building-a-resilient-fallback-pipeline", "markdown": "https://wpnews.pro/news/when-my-ai-api-went-down-building-a-resilient-fallback-pipeline.md", "text": "https://wpnews.pro/news/when-my-ai-api-went-down-building-a-resilient-fallback-pipeline.txt", "jsonld": "https://wpnews.pro/news/when-my-ai-api-went-down-building-a-resilient-fallback-pipeline.jsonld"}}