Preventing Runaway LLM Bills: Building a Resilient AI Worker Queue with Redis, BullMQ, and Model Fallbacks

A developer released resilient-ai-worker-kit, an open-source TypeScript toolkit that pairs BullMQ and Redis to cap runaway LLM API spend. The kit pre-checks estimated tokens against an atomic Redis daily budget counter before dispatch, routes requests through a multi-provider failover chain on 429 or 500 errors, and records actual token usage after each job, with jittered exponential retries in the worker.

Building autonomous agents or AI-driven background tasks is exciting—until a bad retry loop or unexpected 429 rate-limit cascades into a $2,000 API bill overnight. In production, naive API calls with standard try/catch fail for three predictable reasons: In this tutorial, we will construct a production-ready AI Worker Queue in TypeScript using BullMQ and Redis , featuring: php flowchart TD A Incoming Task Request -- B BullMQ Job Queue B -- C{Daily Budget Pre-Check} C -- Exceeds Spend Ceiling -- D Abort / Kill Switch Triggered C -- Within Budget Limit -- E Resilient Multi-Provider Router E -- Primary 429/Timeout -- F Fallback Provider: Groq / Anthropic E -- Primary Success -- G Worker Output Delivered F -- G G -- H Atomic Redis Token Increment Before an expensive prompt is dispatched to an LLM provider, we pre-calculate estimated tokens and verify against an atomic Redis counter: export class BudgetManager { private redis: Redis; private keyPrefix = "ai budget:daily:"; public async canExecute estimatedTokens: number : Promise<{ allowed: boolean; currentUsage: number; reason?: string } { const today = new Date .toISOString .split "T" 0 ; const key = ${this.keyPrefix}${today} ; const current = parseInt await this.redis.get key || "0", 10 ; if current + estimatedTokens MAX DAILY TOKEN BUDGET { return { allowed: false, currentUsage: current, reason: Daily token limit breached: ${current} + ${estimatedTokens} ${MAX DAILY TOKEN BUDGET} , }; } return { allowed: true, currentUsage: current }; } public async recordUsage tokens: number : Promise<number { const today = new Date .toISOString .split "T" 0 ; const key = ${this.keyPrefix}${today} ; const total = await this.redis.incrby key, tokens ; await this.redis.expire key, 172800 ; // 48h TTL return total; } } If your primary provider returns a 429 Too Many Requests or 500 Internal Error , you shouldn't crash the queue. Instead, fall back to secondary high-availability endpoints seamlessly: export class ResilientModelRouter { constructor private providers: IModelProvider {} public async executeWithFailover req: ModelRequest : Promise<ModelResponse { for const provider of this.providers { try { return await provider.execute req ; } catch err: any { console.warn Failover Provider ${provider.name} failed: ${err.message}. Trying next fallback... ; } } throw new Error All fallback providers exhausted for request ${req.id} ; } } We bind the budget check and failover router directly into a BullMQ worker process with jittered exponential retries: js const worker = new Worker<ModelRequest, ModelResponse "resilient-ai-jobs", async job: Job<ModelRequest = { const estimatedTokens = Math.ceil job.data.prompt.length / 4 + 500; // 1. Budget kill switch check const budgetStatus = await budgetManager.canExecute estimatedTokens ; if budgetStatus.allowed { throw new Error BUDGET EXCEEDED ${budgetStatus.reason} ; } // 2. Execute with automatic provider failover const response = await router.executeWithFailover job.data ; // 3. Atomically account for actual token consumption await budgetManager.recordUsage response.tokensUsed ; return response; }, { connection: redisConnection, concurrency: 5 } ; The complete, working source code is open-sourced on GitHub: 👉 GitHub: sidsri14/resilient-ai-worker-kit For enterprise queue dashboards, distributed rate-limiting middleware, and production Docker configs, grab the Enterprise License via Polar.sh https://buy.polar.sh/polar cl 4mrUXvOPjiB4LEIEBsVyJshejMA8oq7XIHPQV163dbE .