Practical Rate Limiting for Free Public AI Endpoints Without Annoying Real Users

A developer outlined a lightweight rate-limiting pattern for free, no-signup AI endpoints that combines a 5-request burst limit per 30 seconds with a 25,000-character hourly budget tracked in Redis, rather than counting requests alone. The approach normalizes IPv6 addresses to /64 prefixes to avoid false positives from rotating host addresses, adds anonymous HttpOnly session tokens, deduplicates identical prompts for 60 seconds via a hash cache, and returns structured 429 responses with a Retry-After header and a frontend countdown. The developer argues these measures curb accidental and automated token waste while keeping the first-visit experience smooth for genuine users.

If you ship a free, no-signup AI web tool, a naive rate limit like "10 requests per minute per IP" almost always fails in both directions. Real users on shared mobile carrier NATs or office Wi-Fi hit false-positive blocks after two clicks, while simple rotating-IP scripts burn through your token budget in minutes. Here is a lightweight, practical rate-limiting pattern that protects LLM endpoints without ruining the first-time experience for genuine visitors. Counting only requests treats a 40-character prompt the same as a 4,000-character prompt. Instead, enforce two lightweight checks in Redis or an in-memory store for single-instance apps : type LimitCheck = { allowed: boolean; retryAfterSec: number; }; type RedisCounter = { incr: k: string = Promise<number ; incrby: k: string, n: number = Promise<number ; expire: k: string, s: number = Promise<void ; ttl: k: string = Promise<number ; }; async function checkDualLimit key: string, charCost: number, redis: RedisCounter : Promise<LimitCheck { const burstKey = rl:burst:${key} ; const budgetKey = rl:chars:${key} ; const burstCount = await redis.incr burstKey ; if burstCount === 1 await redis.expire burstKey, 30 ; if burstCount 5 { const ttl = Math.max 1, await redis.ttl burstKey ; return { allowed: false, retryAfterSec: ttl }; } const totalChars = await redis.incrby budgetKey, charCost ; if totalChars === charCost await redis.expire budgetKey, 3600 ; if totalChars 25000 { const ttl = Math.max 1, await redis.ttl budgetKey ; return { allowed: false, retryAfterSec: ttl }; } return { allowed: true, retryAfterSec: 0 }; } On mobile networks and dual-stack connections, a single user's IPv6 address can rotate across requests, while thousands of legitimate visitors can share one IPv4 exit node. Two small adjustments reduce both abuse and false positives: /64 prefix /128 host address. HttpOnly anonymous session token Every rejected invalid request before your LLM call saves money and keeps your queue fast for real users: maxInputChars limit right in the route handler. normalizedPrompt + mode + locale for 60 seconds. If a user accidentally hits Submit twice—or refreshes immediately—serve the cached response without counting it as a second model call. Origin / Sec-Fetch-Site headers 429 Payloads With Retry-After A plain "Too Many Requests" string leaves users guessing whether the app is broken or permanently blocked. Return a standard Retry-After header plus structured JSON: return Response.json { error: "rate limited", message: "You've sent several requests quickly. Please wait a moment and try again.", retryAfterSec: result.retryAfterSec, }, { status: 429, headers: { "Retry-After": String result.retryAfterSec , }, } ; On the frontend, disable the submit button with a live countdown Try again in 14s instead of letting frustrated users hammer the button and extend their own cooldown. Good rate limiting on public AI endpoints isn't about locking everything down with heavy CAPTCHAs on the first visit. Combining burst + character budgets , /64 IPv6 normalization , short prompt-hash deduplication , and clear 429 countdowns stops most accidental and automated waste while keeping the experience smooth for real visitors. How are you protecting free LLM endpoints in production—token buckets, anonymous session cookies, or Edge middleware?