{"slug": "stop-letting-flaky-apis-crash-your-ai-agents", "title": "Stop Letting Flaky APIs Crash Your AI Agents", "summary": "A developer detailed a production-grade pattern for making AI agents resilient to flaky external APIs, combining exponential backoff, circuit breakers, and graceful fallbacks. The approach wraps every tool call in a defensive pipeline that returns degraded results with metadata, allowing the LLM to adjust its reasoning instead of crashing. The implementation uses the tenacity and circuitbreaker libraries to ensure agents survive rate limits, timeouts, and outages.", "body_md": "*How to combine exponential backoff, circuit breakers, and graceful fallbacks for production-grade agentic workflows.*\n\nAI agents are only as reliable as the tools they invoke. When an LLM decides to search the web, scrape a URL, or fetch database records, it depends entirely on network stability.\n\nIn production, external APIs fail constantly. A sudden surge causes 429 rate limits, a third-party microservice throws a 504 timeout, or a target endpoint goes down entirely. The naive approach—executing raw tool calls directly inside the agent loop—is a ticking time bomb:\n\n``` python\n# The Naive Anti-Pattern: Fragile Tool Execution\ndef execute_agent_tool(tool_name: str, payload: dict):\n    # One 500 error here kills the entire multi-step reasoning chain\n    response = requests.post(f\"https://api.service.internal/{tool_name}\", json=payload)\n    return response.json()\n```\n\nWhen this call breaks, the unhandled exception crashes the runtime. You lose the entire reasoning graph, waste LLM tokens, and degrade the user experience.\n\nTo keep multi-step agents alive, you need a defensive execution pipeline wrapped around every tool. Instead of allowing errors to bubble up and kill the agent, we handle failures across three distinct layers:\n\n```\n[ Agent Core ] \n      │\n      ▼\n┌───────────────────────────────┐\n│     Circuit Breaker Check     │\n│   (Is Primary Service Up?)    │\n└──────────────┬────────────────┘\n       OPEN    │   CLOSED (Healthy)\n       ┌───────┴────────┐\n       ▼                ▼\n┌─────────────┐  ┌───────────────────────────┐\n│  Fallback   │  │ Retry Engine (Backoff)    │\n│  Provider   │  │ └──> Primary API Endpoint │\n└──────┬──────┘  └──────────────┬────────────┘\n       │                        │ SUCCESS\n       │ (Degraded)             │\n       ▼                        ▼\n┌────────────────────────────────────────────┐\n│      Structured Response + Metadata        │\n│   (LLM receives context of degradation)    │\n└────────────────────────────────────────────┘\n```\n\nBy returning a degraded result accompanied by metadata (e.g., `status: \"degraded\", source: \"fallback_cache\"`\n\n), the LLM can adjust its downstream reasoning rather than hallucinating over missing data.\n\nWe combine `tenacity`\n\nfor retry logic with `circuitbreaker`\n\nto isolate failing services. The following production-ready pattern ensures failures are caught and handled before reaching the LLM orchestrator.\n\n``` python\nfrom tenacity import retry, stop_after_attempt, wait_exponential\nfrom circuitbreaker import circuit, CircuitBreakerError\n\n# 1. Protect external call with circuit breaker and exponential backoff\n@circuit(failure_threshold=3, recovery_timeout=60)\n@retry(stop=stop_after_attempt(3), wait=wait_exponential(multiplier=1, min=2, max=10))\ndef fetch_primary_data(query: str) -> dict:\n    resp = requests.get(\"https://api.flaky-service.com/v1/search\", params={\"q\": query}, timeout=3)\n    resp.raise_for_status()\n    return {\"data\": resp.json(), \"source\": \"primary\", \"degraded\": False}\n\n# 2. Resilient fallback wrapper for the agent runtime\ndef execute_tool_safely(query: str) -> dict:\n    try:\n        return fetch_primary_data(query)\n    except (CircuitBreakerError, Exception) as err:\n        # Fallback to internal cache or secondary tool\n        cached_result = local_cache.get(query) or \"No fresh data available.\"\n        return {\n            \"data\": cached_result,\n            \"source\": \"cache_fallback\",\n            \"degraded\": True,\n            \"error_context\": str(err)\n        }\n```\n\nThis snippet ensures three critical guarantees:\n\n`\"Note: Live search is offline. Using cached results from 2 hours ago.\"`\n\n). This keeps the model's responses", "url": "https://wpnews.pro/news/stop-letting-flaky-apis-crash-your-ai-agents", "canonical_source": "https://dev.to/srijan_bhai/stop-letting-flaky-apis-crash-your-ai-agents-a50", "published_at": "2026-08-30 12:22:39+00:00", "updated_at": "2026-08-30 12:52:52.003016+00:00", "lang": "en", "topics": ["ai-agents", "ai-infrastructure", "developer-tools", "ai-products"], "entities": ["tenacity", "circuitbreaker"], "alternates": {"html": "https://wpnews.pro/news/stop-letting-flaky-apis-crash-your-ai-agents", "markdown": "https://wpnews.pro/news/stop-letting-flaky-apis-crash-your-ai-agents.md", "text": "https://wpnews.pro/news/stop-letting-flaky-apis-crash-your-ai-agents.txt", "jsonld": "https://wpnews.pro/news/stop-letting-flaky-apis-crash-your-ai-agents.jsonld"}}