How to Design a Rate Limiter — Token Bucket, Sliding Window & More (System Design Explained)

A system design article explains how to design a rate limiter for APIs, covering algorithms like token bucket and sliding window, their trade-offs, and distributed implementation. The piece emphasizes rate limiting's role in preventing abuse and ensuring fair resource allocation.

Member-only story How to Design a Rate Limiter — Token Bucket, Sliding Window & More System Design Explained Every serious API has a rate limiter in front of it. Without one, a single misbehaving client — or a deliberate attacker — can consume all your capacity and take the service down for everyone. Rate limiting sounds like a small utility feature. It’s actually a genuinely interesting design problem, it appears constantly in system design interviews, and the algorithm you choose has real consequences for how your API behaves under pressure. Let’s design one properly — covering all the major algorithms, their trade-offs, and how to run rate limiting across distributed servers. What a Rate Limiter Does A rate limiter enforces rules like: - A user can make at most 100 API requests per minute - An IP address can attempt login at most 5 times per hour - A free-tier client can call the AI endpoint at most 10 times per day When the limit is exceeded, requests are rejected — typically with HTTP 429 Too Many Requests and a header indicating when to retry: HTTP/1.1 429 Too Many RequestsRetry-After: 30X-RateLimit-Limit: 100X-RateLimit-Remaining: 0X-RateLimit-Reset: 1719475200