{"slug": "ai-rate-limiting-for-llm-calls-in-typescript", "title": "AI Rate Limiting for LLM Calls in TypeScript", "summary": "A developer demonstrated that adding a local concurrency limiter to LLM calls in TypeScript, using the VernLLM library, eliminated provider 429 errors in a 50-call burst test. With retries alone, the test sent 130 requests, collected 110 rate-limit rejections and lost 30 calls; with rateLimit maxConcurrent set to 10, all 50 calls succeeded with zero 429s. The library offers requests-per-minute, tokens-per-minute and concurrency buckets, FIFO queuing with configurable timeouts, AIMD-based adaptive ceilings and per-target limiters for fallbacks.", "body_md": "You fire 50 LLM calls at once. A batch job, a fan out over documents, a busy hour. The provider only lets 10 through at a time, so the rest come back as `429 Too Many Requests`.\n\nThe usual fix is to retry 429s with backoff. That helps, but notice what it means: every retry is a request you sent after you already got rate limited. You are reacting to a limit you could have stayed under.\n\nThis post shows the difference with real numbers and how to rate limit LLM calls in TypeScript with [VernLLM](https://vernllm.dev), before the provider says no.\n\nI ran a 50 call burst against a scripted provider that accepts 10 requests in flight and rejects the rest with 429. Both runs used `maxRetries: 2`.\n\n|  | Retries only | `rateLimit: { maxConcurrent: 10 }` | \n|---|---|---|\n| Calls answered | 20 | 50 | \n| Calls failed | 30 | 0 | \n| Provider 429s | 110 | 0 | \n| Requests sent | 130 | 50 | \n\nRetries alone sent 130 requests, collected 110 rejections, and still lost 30 calls. With a local limit, every call waited its turn and went out once.\n\n```\nnpm i vern-llm openai\npython\nimport OpenAI from 'openai';\nimport { VernLLM } from 'vern-llm';\nimport { fromOpenAI } from 'vern-llm/adapters';\n\nconst llm = new VernLLM({\n  client: fromOpenAI(new OpenAI({ apiKey: process.env.OPENAI_API_KEY })),\n  model: 'gpt-6.2-sol',\n  rateLimit: {\n    requestsPerMinute: 500,\n    tokensPerMinute: 200_000,\n    maxConcurrent: 20,\n  },\n});\n\nconst results = await Promise.all(tickets.map((t) => llm.call({ userContent: t.text })));\n```\n\n`rateLimit` has three buckets, each optional:\n\nA call goes out only when every configured bucket has room. Otherwise it waits.\n\nCalls that cannot go out yet wait in strict first in, first out order. A large call is never starved by smaller ones behind it.\n\nYou decide how long a call may wait and how deep the queue may get:\n\n```\nrateLimit: {\n  requestsPerMinute: 60,\n  maxQueueMs: 10_000, // default 30000\n  maxQueueSize: 50,   // default unbounded\n},\n```\n\nWhen a call gives up waiting, it fails with a `rate_limited` error and a code you can branch on (`rate_limit_queue_timeout` or `rate_limit_queue_full`). These never reached the provider, so VernLLM does not retry them. A call too big to ever fit under `tokensPerMinute` fails right away with `rate_limit_capacity_exceeded` instead of blocking the queue forever.\n\nCapacity is taken per attempt, not per call. A retry is a real request against the same provider limit, so it waits in the same queue as everything else. That is what keeps retries from turning one 429 into a storm of them.\n\nYou often do not know the provider's real limit, or it changes with your tier and their load. `aimd` lets the requests ceiling adjust itself: it grows a little on every success and shrinks on a real 429.\n\n```\nrateLimit: {\n  requestsPerMinute: 500,\n  aimd: { increaseBy: 10, decreaseFactor: 0.5, minCapacity: 50, maxCapacity: 1000 },\n},\n```\n\nIt shrinks at most once per 60 seconds, so a burst of 429s from calls already in flight counts as one overload. With `proactiveFloor`, adapters that can read rate limit headers shrink the ceiling when the provider reports low remaining capacity, before a 429 happens.\n\nA 429 that still gets through keeps its own code, `provider_rate_limited`, so you can tell a real provider limit apart from VernLLM's own queue in your logs.\n\nWith fallback, each target gets its own limiter. A burst against a struggling primary cannot use up the slots a healthy backup needs:\n\n``` js\nconst llm = new VernLLM({\n  client: fromOpenAI(openai),\n  model: 'gpt-6.2-sol',\n  rateLimit: { maxConcurrent: 50 },\n  fallback: {\n    client: fromGemini(gemini),\n    model: 'gemini-4-neon',\n    rateLimit: { maxConcurrent: 10 },\n  },\n});\n```\n\nThe built in limiter lives in one process. If you run several instances, pass your own `RateLimiterAdapter`, backed by Redis or anything else, so every instance shares one budget against the provider.\n\nRetrying 429s is damage control. A local limit keeps you under the provider's ceiling in the first place: queue in order, count every retry, adapt when the provider pushes back.\n\n```\nnpm i vern-llm\n```\n\nDocs: [Rate limiting](https://vernllm.dev/docs/core/rate-limiting)", "url": "https://wpnews.pro/news/ai-rate-limiting-for-llm-calls-in-typescript", "canonical_source": "https://dev.to/lakbud/ai-rate-limiting-for-llm-calls-in-typescript-3mfo", "published_at": "2026-11-20 18:10:06+00:00", "updated_at": "2026-11-20 18:14:10.354470+00:00", "lang": "en", "topics": ["large-language-models", "ai-tools", "developer-tools", "ai-infrastructure"], "entities": ["VernLLM", "OpenAI", "Gemini"], "also_reported_by": [], "takeaway": "A local concurrency limiter (maxConcurrent: 10) answered all 50 burst calls with zero 429s, versus 30 failures and 110 rejections when relying on retries alone.", "recommendation": {"advice": "Queue LLM calls locally with a concurrency and rate limiter before they hit the provider, and count retries against the same budget to avoid 429 storms.", "playbook": "llm-cost", "playbook_url": "https://wpnews.pro/playbooks/llm-cost"}, "alternates": {"html": "https://wpnews.pro/news/ai-rate-limiting-for-llm-calls-in-typescript", "markdown": "https://wpnews.pro/news/ai-rate-limiting-for-llm-calls-in-typescript.md", "text": "https://wpnews.pro/news/ai-rate-limiting-for-llm-calls-in-typescript.txt", "jsonld": "https://wpnews.pro/news/ai-rate-limiting-for-llm-calls-in-typescript.jsonld"}}