{"slug": "rate-limiting-a-practical-guide-from-scratch", "title": "Rate Limiting: A Practical Guide from Scratch", "summary": "Rate limiting is essential for stable API deployments, with three main algorithms available: fixed window counter, sliding window log, and token bucket. The fixed window counter suffers from a boundary problem that can double throughput in bursts, while the sliding window log provides smoother control but consumes more memory. The token bucket algorithm is recommended for real-world AI workflow integrations due to its memory efficiency and ability to handle bursts while maintaining a strict long-term average.", "body_md": "# Rate Limiting: A Practical Guide from Scratch\n\nAPI crashes usually happen because one rogue client or a sudden traffic spike slams the backend, so implementing rate limiting is non-negotiable for any stable deployment. Depending on your traffic patterns, some algorithms handle bursts better than others.\n\nThis is the most basic approach. You track requests within a rigid time block (e.g., 60 seconds). If the counter hits the limit, further requests are blocked until the next window starts.\n\nTo fix the boundary spike, the sliding window log keeps a precise timestamp of every request. It only counts requests that happened within the last X seconds from the current millisecond.\n\nThis is my go-to for real-world AI workflow integrations. Imagine a bucket that refills with tokens at a constant rate. Requests consume tokens. If the bucket is empty, the request is rejected.\n\n## Strategy 1: Fixed Window Counter\n\nThis is the most basic approach. You track requests within a rigid time block (e.g., 60 seconds). If the counter hits the limit, further requests are blocked until the next window starts.\n\n``` python\nimport time\nfrom collections import defaultdict\n\nclass FixedWindowLimiter:\n def __init__(self, max_requests=10, window_seconds=60):\n self.max_requests = max_requests\n self.window_seconds = window_seconds\n self.requests = defaultdict(list)\n\n def allow_request(self, user_id):\n now = time.time()\n window_start = now - self.window_seconds\n # Remove old entries\n self.requests[user_id] = [t for t in self.requests[user_id] if t > window_start]\n if len(self.requests[user_id]) >= self.max_requests:\n return False\n self.requests[user_id].append(now)\n return True\n```\n\nThe main issue here is the \"boundary problem.\" A user could exhaust their limit at the very end of window A and immediately use another full quota at the start of window B, effectively doubling their throughput in a short burst.\n\n## Strategy 2: Sliding Window Log\n\nTo fix the boundary spike, the sliding window log keeps a precise timestamp of every request. It only counts requests that happened within the last X seconds from the current millisecond.\n\n``` python\nimport time\nfrom collections import defaultdict\n\nclass SlidingWindowLimiter:\n def __init__(self, max_requests=10, window_seconds=60):\n self.max_requests = max_requests\n self.window_seconds = window_seconds\n self.logs = defaultdict(list)\n\n def allow_request(self, user_id):\n now = time.time()\n cutoff = now - self.window_seconds\n # Remove timestamps outside the window\n self.logs[user_id] = [t for t in self.logs[user_id] if t > cutoff]\n if len(self.logs[user_id]) >= self.max_requests:\n return False\n self.logs[user_id].append(now)\n return True\n```\n\nThis is much smoother, but memory is the trade-off. Storing every single timestamp for millions of users will eat up your RAM quickly.\n\n## Strategy 3: Token Bucket\n\nThis is my go-to for real-world AI workflow integrations. Imagine a bucket that refills with tokens at a constant rate. Requests consume tokens. If the bucket is empty, the request is rejected.\n\n``` python\nimport time\n\nclass TokenBucket:\n def __init__(self, capacity=10, refill_rate=1):\n self.capacity = capacity\n self.refill_rate = refill_rate # tokens per second\n self.tokens = capacity\n self.last_refill = time.time()\n\n def allow_request(self):\n now = time.time()\n elapsed = now - self.last_refill\n self.tokens = min(self.capacity, self.tokens + elapsed * self.refill_rate)\n self.last_refill = now\n if self.tokens >= 1:\n self.tokens -= 1\n return True\n return False\n```\n\nIt's memory-efficient and handles bursts beautifully while maintaining a strict long-term average.\n\n## Implementation Tips\n\n**Distributed State:** In-memory lists only work for single-instance apps. For any real production deployment, move your counters to Redis.**Client Communication:** Don't just return a 429 error. Always include`X-RateLimit-Limit`\n\n,`X-RateLimit-Remaining`\n\n, and`X-RateLimit-Reset`\n\nheaders so the client knows when to back off.**Cleanup:** If you use in-memory maps, you must implement a TTL or a cleanup job, otherwise, your memory usage will grow linearly with your user base.\n\n[Next Qwen3 Text Encoder: Fixing FP8 and GGUF Crashes →](/en/threads/3769/)\n\n## All Replies （3）\n\nM\n\nSaved my app once when a buggy loop in my frontend spammed the API. Essential stuff.\n\n0\n\nA\n\nDon't forget to add a \"Retry-After\" header so clients know when to try again.\n\n0\n\nN\n\nDo you recommend using Redis for the counter or keeping it in-memory for lower latency?\n\n0", "url": "https://wpnews.pro/news/rate-limiting-a-practical-guide-from-scratch", "canonical_source": "https://promptcube3.com/en/threads/3781/", "published_at": "2026-07-26 17:45:54+00:00", "updated_at": "2026-07-26 18:09:19.711881+00:00", "lang": "en", "topics": ["developer-tools"], "entities": [], "alternates": {"html": "https://wpnews.pro/news/rate-limiting-a-practical-guide-from-scratch", "markdown": "https://wpnews.pro/news/rate-limiting-a-practical-guide-from-scratch.md", "text": "https://wpnews.pro/news/rate-limiting-a-practical-guide-from-scratch.txt", "jsonld": "https://wpnews.pro/news/rate-limiting-a-practical-guide-from-scratch.jsonld"}}