{"slug": "do-not-retry-an-llm-rate-limit-reschedule-it", "title": "Do Not Retry an LLM Rate Limit, Reschedule It", "summary": "A developer has documented a queue-handling pattern that treats LLM provider rate limits as scheduling signals rather than failures, using BullMQ's rateLimit mechanism and a RateLimitError to reschedule jobs instead of consuming retry attempts. The approach distinguishes per-minute limits from daily quotas by parsing the 429 error body and honoring the Retry-After header, and a companion KeyPool class rotates Gemini API keys with per-key cooldowns. In the LeadForge AI case, the default five-attempt exponential backoff exhausted all retries in about thirty seconds and dead-lettered a job while hours of daily quota remained.", "body_md": "A rate limit is not a failure, so do not spend retry attempts on it. Work out which limit you hit, then delay the job until that limit resets. Retrying a daily quota error immediately only fills your dead letter queue while hours of quota remain.\n\nLLM providers enforce several limits at once. Requests per minute and tokens per minute reset within a minute. A daily quota resets once a day. Both can come back as the same HTTP status, 429, and that is where the trouble starts.\n\nIf a job hits the per minute limit, waiting 30 seconds fixes it. If it hits the daily quota, waiting 30 seconds fixes nothing, and neither does waiting an hour. The correct delay depends on which limit tripped, and the status code alone does not tell you. Read the error body, since most providers say which quota was exceeded, and check your provider’s documentation for when each one resets.\n\nA queue with five attempts and exponential backoff is a reasonable default for network blips. Pointed at a daily quota it is a disaster. In LeadForge AI the default behaviour burned all five retries within about thirty seconds and then dead lettered the job, while hours of quota window still lay ahead. The job was fine. The system had simply given up on it far too early.\n\nThe fix was a change in how I think about it. A rate limit says “not now”, and a failure says “this did not work”. Only the second deserves a retry attempt.\n\nBullMQ has a mechanism for exactly this. Inside the worker you call rateLimit with a delay and then throw RateLimitError. The job goes back to waiting without being counted as a failed attempt.\n\n``` js\nimport { Worker } from 'bullmq'\n\nconst worker = new Worker(\n  'enrich-leads',\n  async (job) => {\n    try {\n      return await callModel(job.data)\n    } catch (error) {\n      const waitMs = cooldownFor(error)\n      if (waitMs) {\n        await worker.rateLimit(waitMs)\n        throw Worker.RateLimitError()\n      }\n      throw error // real failures still use normal retries\n    }\n  },\n  { connection }\n)\n```\n\nNotice that rateLimit applies to the worker’s queue, so every job on it waits, not just the one that failed. That is what you want when the quota is shared by all jobs. If one job is the problem, look at moving that single job to the delayed set instead.\n\nThe function that matters is cooldownFor. It reads the error and returns how long to wait, or nothing if the error is not a rate limit at all.\n\n```\nfunction cooldownFor(error) {\n  if (error.status !== 429) return 0\n\n  // Honour the server when it tells you how long to wait.\n  const retryAfter = Number(error.headers?.['retry-after'])\n  if (retryAfter) return retryAfter * 1000\n\n  // Daily quota: wait until the provider's reset, not a few seconds.\n  if (/per day|daily/i.test(error.message)) return msUntilQuotaReset()\n\n  // Per minute limit: a minute is enough.\n  return 60_000\n}\n```\n\nThe message patterns above are examples. Log a few real errors from your provider and match on what they actually say.\n\nNoteMind uses a pool of Gemini keys, round robin, with a cooldown per key that respects the difference between daily and per minute limits. A key that hit its per minute limit should come back soon. A key that hit its daily limit should sit out until the reset. Here is a minimal version of that idea.\n\n``` js\nexport class KeyPool {\n  constructor(keys) {\n    this.slots = keys.map((key) => ({ key, availableAt: 0 }))\n    this.cursor = 0\n  }\n\n  next(now = Date.now()) {\n    for (let i = 0; i < this.slots.length; i++) {\n      const index = (this.cursor + i) % this.slots.length\n      const slot = this.slots[index]\n      if (slot.availableAt <= now) {\n        this.cursor = (index + 1) % this.slots.length\n        return slot\n      }\n    }\n    return null // every key is cooling down\n  }\n\n  cool(slot, ms) {\n    slot.availableAt = Date.now() + ms\n  }\n\n  earliestAvailable() {\n    return Math.min(...this.slots.map((slot) => slot.availableAt))\n  }\n}\n```\n\nWhen next returns null you do not fail anything. You pass the gap until earliestAvailable to the queue and reschedule, exactly as before.\n\nLeadForge AI runs verification, scoring and outreach through BullMQ workers on Redis, and the full design is in the [LeadForge AI case study](https://ahmadhsn1.github.io/work/leadforge-ai/). NoteMind’s key pool is covered in the [NoteMind case study](https://ahmadhsn1.github.io/work/notemind/).\n\n*Originally published at [ahmadhsn1.github.io](https://ahmadhsn1.github.io/blog/llm-rate-limits-bullmq-reschedule/).*", "url": "https://wpnews.pro/news/do-not-retry-an-llm-rate-limit-reschedule-it", "canonical_source": "https://dev.to/ahmad_hassan_6433415b38c6/do-not-retry-an-llm-rate-limit-reschedule-it-38d3", "published_at": "2026-10-15 11:39:11+00:00", "updated_at": "2026-10-15 11:48:55.809268+00:00", "lang": "en", "topics": ["large-language-models", "ai-infrastructure", "mlops", "developer-tools", "ai-agents"], "entities": ["BullMQ", "LeadForge AI", "NoteMind", "Gemini"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/do-not-retry-an-llm-rate-limit-reschedule-it", "markdown": "https://wpnews.pro/news/do-not-retry-an-llm-rate-limit-reschedule-it.md", "text": "https://wpnews.pro/news/do-not-retry-an-llm-rate-limit-reschedule-it.txt", "jsonld": "https://wpnews.pro/news/do-not-retry-an-llm-rate-limit-reschedule-it.jsonld"}}