{"slug": "preventing-runaway-llm-bills-building-a-resilient-ai-worker-queue-with-redis-and", "title": "Preventing Runaway LLM Bills: Building a Resilient AI Worker Queue with Redis, BullMQ, and Model Fallbacks", "summary": "A developer released resilient-ai-worker-kit, an open-source TypeScript toolkit that pairs BullMQ and Redis to cap runaway LLM API spend. The kit pre-checks estimated tokens against an atomic Redis daily budget counter before dispatch, routes requests through a multi-provider failover chain on 429 or 500 errors, and records actual token usage after each job, with jittered exponential retries in the worker.", "body_md": "Building autonomous agents or AI-driven background tasks is exciting—until a bad retry loop or unexpected 429 rate-limit cascades into a $2,000 API bill overnight.\n\nIn production, naive API calls with standard `try/catch` fail for three predictable reasons:\n\nIn this tutorial, we will construct a production-ready **AI Worker Queue** in TypeScript using **BullMQ** and **Redis**, featuring:\n\n``` php\nflowchart TD\n    A[Incoming Task Request] --> B[BullMQ Job Queue]\n    B --> C{Daily Budget Pre-Check}\n    C -- Exceeds Spend Ceiling --> D[Abort / Kill Switch Triggered]\n    C -- Within Budget Limit --> E[Resilient Multi-Provider Router]\n    E -- Primary 429/Timeout --> F[Fallback Provider: Groq / Anthropic]\n    E -- Primary Success --> G[Worker Output Delivered]\n    F --> G\n    G --> H[Atomic Redis Token Increment]\n```\n\nBefore an expensive prompt is dispatched to an LLM provider, we pre-calculate estimated tokens and verify against an atomic Redis counter:\n\n```\nexport class BudgetManager {\n  private redis: Redis;\n  private keyPrefix = \"ai_budget:daily:\";\n\n  public async canExecute(estimatedTokens: number): Promise<{ allowed: boolean; currentUsage: number; reason?: string }> {\n    const today = new Date().toISOString().split(\"T\")[0];\n    const key = `${this.keyPrefix}${today}`;\n    const current = parseInt((await this.redis.get(key)) || \"0\", 10);\n\n    if (current + estimatedTokens > MAX_DAILY_TOKEN_BUDGET) {\n      return {\n        allowed: false,\n        currentUsage: current,\n        reason: `Daily token limit breached: ${current} + ${estimatedTokens} > ${MAX_DAILY_TOKEN_BUDGET}`,\n      };\n    }\n\n    return { allowed: true, currentUsage: current };\n  }\n\n  public async recordUsage(tokens: number): Promise<number> {\n    const today = new Date().toISOString().split(\"T\")[0];\n    const key = `${this.keyPrefix}${today}`;\n    const total = await this.redis.incrby(key, tokens);\n    await this.redis.expire(key, 172800); // 48h TTL\n    return total;\n  }\n}\n```\n\nIf your primary provider returns a `429 Too Many Requests` or `500 Internal Error`, you shouldn't crash the queue. Instead, fall back to secondary high-availability endpoints seamlessly:\n\n```\nexport class ResilientModelRouter {\n  constructor(private providers: IModelProvider[]) {}\n\n  public async executeWithFailover(req: ModelRequest): Promise<ModelResponse> {\n    for (const provider of this.providers) {\n      try {\n        return await provider.execute(req);\n      } catch (err: any) {\n        console.warn(`[Failover] Provider ${provider.name} failed: ${err.message}. Trying next fallback...`);\n      }\n    }\n    throw new Error(`All fallback providers exhausted for request [${req.id}]`);\n  }\n}\n```\n\nWe bind the budget check and failover router directly into a BullMQ worker process with jittered exponential retries:\n\n``` js\nconst worker = new Worker<ModelRequest, ModelResponse>(\n  \"resilient-ai-jobs\",\n  async (job: Job<ModelRequest>) => {\n    const estimatedTokens = Math.ceil((job.data.prompt.length / 4)) + 500;\n\n    // 1. Budget kill switch check\n    const budgetStatus = await budgetManager.canExecute(estimatedTokens);\n    if (!budgetStatus.allowed) {\n      throw new Error(`[BUDGET_EXCEEDED] ${budgetStatus.reason}`);\n    }\n\n    // 2. Execute with automatic provider failover\n    const response = await router.executeWithFailover(job.data);\n\n    // 3. Atomically account for actual token consumption\n    await budgetManager.recordUsage(response.tokensUsed);\n\n    return response;\n  },\n  { connection: redisConnection, concurrency: 5 }\n);\n```\n\nThe complete, working source code is open-sourced on GitHub:\n\n👉 **GitHub: sidsri14/resilient-ai-worker-kit**\n\nFor enterprise queue dashboards, distributed rate-limiting middleware, and production Docker configs, grab the **Enterprise License** via [Polar.sh](https://buy.polar.sh/polar_cl_4mrUXvOPjiB4LEIEBsVyJshejMA8oq7XIHPQV163dbE).", "url": "https://wpnews.pro/news/preventing-runaway-llm-bills-building-a-resilient-ai-worker-queue-with-redis-and", "canonical_source": "https://dev.to/sidsri14/preventing-runaway-llm-bills-building-a-resilient-ai-worker-queue-with-redis-bullmq-and-model-23eh", "published_at": "2026-11-05 16:45:27+00:00", "updated_at": "2026-11-05 16:49:24.572497+00:00", "lang": "en", "topics": ["ai-agents", "ai-infrastructure", "mlops", "developer-tools", "large-language-models"], "entities": ["BullMQ", "Redis", "Groq", "Anthropic", "GitHub", "sidsri14"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/preventing-runaway-llm-bills-building-a-resilient-ai-worker-queue-with-redis-and", "markdown": "https://wpnews.pro/news/preventing-runaway-llm-bills-building-a-resilient-ai-worker-queue-with-redis-and.md", "text": "https://wpnews.pro/news/preventing-runaway-llm-bills-building-a-resilient-ai-worker-queue-with-redis-and.txt", "jsonld": "https://wpnews.pro/news/preventing-runaway-llm-bills-building-a-resilient-ai-worker-queue-with-redis-and.jsonld"}}