{"slug": "practical-rate-limiting-for-free-public-ai-endpoints-without-annoying-real-users", "title": "Practical Rate Limiting for Free Public AI Endpoints Without Annoying Real Users", "summary": "A developer outlined a lightweight rate-limiting pattern for free, no-signup AI endpoints that combines a 5-request burst limit per 30 seconds with a 25,000-character hourly budget tracked in Redis, rather than counting requests alone. The approach normalizes IPv6 addresses to /64 prefixes to avoid false positives from rotating host addresses, adds anonymous HttpOnly session tokens, deduplicates identical prompts for 60 seconds via a hash cache, and returns structured 429 responses with a Retry-After header and a frontend countdown. The developer argues these measures curb accidental and automated token waste while keeping the first-visit experience smooth for genuine users.", "body_md": "If you ship a free, no-signup AI web tool, a naive rate limit like \"10 requests per minute per IP\" almost always fails in both directions.\n\nReal users on shared mobile carrier NATs or office Wi-Fi hit false-positive blocks after two clicks, while simple rotating-IP scripts burn through your token budget in minutes.\n\nHere is a lightweight, practical rate-limiting pattern that protects LLM endpoints without ruining the first-time experience for genuine visitors.\n\nCounting only *requests* treats a 40-character prompt the same as a 4,000-character prompt. Instead, enforce two lightweight checks in Redis (or an in-memory store for single-instance apps):\n\n```\ntype LimitCheck = {\n  allowed: boolean;\n  retryAfterSec: number;\n};\n\ntype RedisCounter = {\n  incr: (k: string) => Promise<number>;\n  incrby: (k: string, n: number) => Promise<number>;\n  expire: (k: string, s: number) => Promise<void>;\n  ttl: (k: string) => Promise<number>;\n};\n\nasync function checkDualLimit(\n  key: string,\n  charCost: number,\n  redis: RedisCounter\n): Promise<LimitCheck> {\n  const burstKey = `rl:burst:${key}`;\n  const budgetKey = `rl:chars:${key}`;\n\n  const burstCount = await redis.incr(burstKey);\n  if (burstCount === 1) await redis.expire(burstKey, 30);\n\n  if (burstCount > 5) {\n    const ttl = Math.max(1, await redis.ttl(burstKey));\n    return { allowed: false, retryAfterSec: ttl };\n  }\n\n  const totalChars = await redis.incrby(budgetKey, charCost);\n  if (totalChars === charCost) await redis.expire(budgetKey, 3600);\n\n  if (totalChars > 25000) {\n    const ttl = Math.max(1, await redis.ttl(budgetKey));\n    return { allowed: false, retryAfterSec: ttl };\n  }\n\n  return { allowed: true, retryAfterSec: 0 };\n}\n```\n\nOn mobile networks and dual-stack connections, a single user's IPv6 address can rotate across requests, while thousands of legitimate visitors can share one IPv4 exit node.\n\nTwo small adjustments reduce both abuse and false positives:\n\n`/64` prefix`/128` host address.`HttpOnly` anonymous session token\nEvery rejected invalid request *before* your LLM call saves money and keeps your queue fast for real users:\n\n`maxInputChars` limit right in the route handler.`(normalizedPrompt + mode + locale)` for 60 seconds. If a user accidentally hits Submit twice—or refreshes immediately—serve the cached response without counting it as a second model call.`Origin` / `Sec-Fetch-Site` headers`429` Payloads With `Retry-After`\nA plain `\"Too Many Requests\"` string leaves users guessing whether the app is broken or permanently blocked. Return a standard `Retry-After` header plus structured JSON:\n\n```\nreturn Response.json(\n  {\n    error: \"rate_limited\",\n    message: \"You've sent several requests quickly. Please wait a moment and try again.\",\n    retryAfterSec: result.retryAfterSec,\n  },\n  {\n    status: 429,\n    headers: {\n      \"Retry-After\": String(result.retryAfterSec),\n    },\n  }\n);\n```\n\nOn the frontend, disable the submit button with a live countdown (`Try again in 14s`) instead of letting frustrated users hammer the button and extend their own cooldown.\n\nGood rate limiting on public AI endpoints isn't about locking everything down with heavy CAPTCHAs on the first visit. Combining **burst + character budgets**, **`/64` IPv6 normalization**, **short prompt-hash deduplication**, and **clear `429` countdowns** stops most accidental and automated waste while keeping the experience smooth for real visitors.\n\nHow are you protecting free LLM endpoints in production—token buckets, anonymous session cookies, or Edge middleware?", "url": "https://wpnews.pro/news/practical-rate-limiting-for-free-public-ai-endpoints-without-annoying-real-users", "canonical_source": "https://dev.to/flort/practical-rate-limiting-for-free-public-ai-endpoints-without-annoying-real-users-1kj3", "published_at": "2026-11-15 08:17:35+00:00", "updated_at": "2026-11-15 08:20:39.370926+00:00", "lang": "en", "topics": ["ai-infrastructure", "ai-tools", "developer-tools", "ai-agents"], "entities": ["Redis"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/practical-rate-limiting-for-free-public-ai-endpoints-without-annoying-real-users", "markdown": "https://wpnews.pro/news/practical-rate-limiting-for-free-public-ai-endpoints-without-annoying-real-users.md", "text": "https://wpnews.pro/news/practical-rate-limiting-for-free-public-ai-endpoints-without-annoying-real-users.txt", "jsonld": "https://wpnews.pro/news/practical-rate-limiting-for-free-public-ai-endpoints-without-annoying-real-users.jsonld"}}