Member-only story
Every serious API has a rate limiter in front of it. Without one, a single misbehaving client — or a deliberate attacker — can consume all your capacity and take the service down for everyone.
Rate limiting sounds like a small utility feature. It’s actually a genuinely interesting design problem, it appears constantly in system design interviews, and the algorithm you choose has real consequences for how your API behaves under pressure.
Let’s design one properly — covering all the major algorithms, their trade-offs, and how to run rate limiting across distributed servers.
What a Rate Limiter Does #
A rate limiter enforces rules like:
- A user can make at most 100 API requests per minute
- An IP address can attempt login at most 5 times per hour
- A free-tier client can call the AI endpoint at most 10 times per day
When the limit is exceeded, requests are rejected — typically with HTTP 429 Too Many Requests
and a header indicating when to retry:
HTTP/1.1 429 Too Many RequestsRetry-After: 30X-RateLimit-Limit: 100X-RateLimit-Remaining: 0X-RateLimit-Reset: 1719475200