AI Rate Limiting for LLM Calls in TypeScript A developer demonstrated that adding a local concurrency limiter to LLM calls in TypeScript, using the VernLLM library, eliminated provider 429 errors in a 50-call burst test. With retries alone, the test sent 130 requests, collected 110 rate-limit rejections and lost 30 calls; with rateLimit maxConcurrent set to 10, all 50 calls succeeded with zero 429s. The library offers requests-per-minute, tokens-per-minute and concurrency buckets, FIFO queuing with configurable timeouts, AIMD-based adaptive ceilings and per-target limiters for fallbacks. You fire 50 LLM calls at once. A batch job, a fan out over documents, a busy hour. The provider only lets 10 through at a time, so the rest come back as 429 Too Many Requests . The usual fix is to retry 429s with backoff. That helps, but notice what it means: every retry is a request you sent after you already got rate limited. You are reacting to a limit you could have stayed under. This post shows the difference with real numbers and how to rate limit LLM calls in TypeScript with VernLLM https://vernllm.dev , before the provider says no. I ran a 50 call burst against a scripted provider that accepts 10 requests in flight and rejects the rest with 429. Both runs used maxRetries: 2 . | | Retries only | rateLimit: { maxConcurrent: 10 } | |---|---|---| | Calls answered | 20 | 50 | | Calls failed | 30 | 0 | | Provider 429s | 110 | 0 | | Requests sent | 130 | 50 | Retries alone sent 130 requests, collected 110 rejections, and still lost 30 calls. With a local limit, every call waited its turn and went out once. npm i vern-llm openai python import OpenAI from 'openai'; import { VernLLM } from 'vern-llm'; import { fromOpenAI } from 'vern-llm/adapters'; const llm = new VernLLM { client: fromOpenAI new OpenAI { apiKey: process.env.OPENAI API KEY } , model: 'gpt-6.2-sol', rateLimit: { requestsPerMinute: 500, tokensPerMinute: 200 000, maxConcurrent: 20, }, } ; const results = await Promise.all tickets.map t = llm.call { userContent: t.text } ; rateLimit has three buckets, each optional: A call goes out only when every configured bucket has room. Otherwise it waits. Calls that cannot go out yet wait in strict first in, first out order. A large call is never starved by smaller ones behind it. You decide how long a call may wait and how deep the queue may get: rateLimit: { requestsPerMinute: 60, maxQueueMs: 10 000, // default 30000 maxQueueSize: 50, // default unbounded }, When a call gives up waiting, it fails with a rate limited error and a code you can branch on rate limit queue timeout or rate limit queue full . These never reached the provider, so VernLLM does not retry them. A call too big to ever fit under tokensPerMinute fails right away with rate limit capacity exceeded instead of blocking the queue forever. Capacity is taken per attempt, not per call. A retry is a real request against the same provider limit, so it waits in the same queue as everything else. That is what keeps retries from turning one 429 into a storm of them. You often do not know the provider's real limit, or it changes with your tier and their load. aimd lets the requests ceiling adjust itself: it grows a little on every success and shrinks on a real 429. rateLimit: { requestsPerMinute: 500, aimd: { increaseBy: 10, decreaseFactor: 0.5, minCapacity: 50, maxCapacity: 1000 }, }, It shrinks at most once per 60 seconds, so a burst of 429s from calls already in flight counts as one overload. With proactiveFloor , adapters that can read rate limit headers shrink the ceiling when the provider reports low remaining capacity, before a 429 happens. A 429 that still gets through keeps its own code, provider rate limited , so you can tell a real provider limit apart from VernLLM's own queue in your logs. With fallback, each target gets its own limiter. A burst against a struggling primary cannot use up the slots a healthy backup needs: js const llm = new VernLLM { client: fromOpenAI openai , model: 'gpt-6.2-sol', rateLimit: { maxConcurrent: 50 }, fallback: { client: fromGemini gemini , model: 'gemini-4-neon', rateLimit: { maxConcurrent: 10 }, }, } ; The built in limiter lives in one process. If you run several instances, pass your own RateLimiterAdapter , backed by Redis or anything else, so every instance shares one budget against the provider. Retrying 429s is damage control. A local limit keeps you under the provider's ceiling in the first place: queue in order, count every retry, adapt when the provider pushes back. npm i vern-llm Docs: Rate limiting https://vernllm.dev/docs/core/rate-limiting