How to Handle API Rate Limits in Python Like a Pro (with Exponential Backoff)

A developer building an AI backend that calls the Gemini API described how to stop server crashes from 429 Too Many Requests errors by implementing exponential backoff in Python using the tenacity library. The approach retries failed requests with waits of 2, 4, then 8 seconds capped at 10 seconds, gives up after five attempts, and can honor a Retry-After header when an API provides one.

If you are building applications that rely on external APIs like OpenAI, Gemini, GitHub, or even just fetching job data , you have probably run into the dreaded 429 Too Many Requests error. When I was building my AI backend, my server kept crashing because I was hammering the Gemini API too fast. If you just write a try/except block and instantly try again, you will get banned. You need Exponential Backoff. Here is how it works and how to build it in Python. What is Exponential Backoff? Instead of retrying a failed request immediately, you wait a little bit. If it fails again, you wait twice as long. If it fails again, you wait twice as long as that. · Attempt 1 fails - Wait 1 second. · Attempt 2 fails - Wait 2 seconds. · Attempt 3 fails - Wait 4 seconds. · Attempt 4 fails - Wait 8 seconds. This gives the API server time to breathe and clear its queue, making your request much more likely to succeed on the next try. The Python Code Using the tenacity Library You could write this logic from scratch using a while loop, but why do that when Python has an incredible library called tenacity? First, install it: pip install tenacity Now, let's build a robust API request function: python import httpx from tenacity import retry, stop after attempt, wait exponential, retry if exception type We only want to retry if we hit a 429 Rate Limit or 500 Server Error def is rate limit error exception : return isinstance exception, httpx.HTTPStatusError and exception.response.status code in 429, 500 @retry retry=retry if exception type httpx.HTTPStatusError , wait=wait exponential multiplier=1, min=2, max=10 , stop=stop after attempt 5 async def fetch external data url: str : print f"Attempting to fetch: {url}" async with httpx.AsyncClient as client: response = await client.get url, timeout=10.0 Raise an exception if the status code is 4xx or 5xx response.raise for status return response.json What does this code do? · wait exponential multiplier=1, min=2, max=10 : This is the magic. It will wait 2s, then 4s, then 8s, capping at 10s so you don't wait forever. · stop after attempt 5 : It will try 5 times before finally giving up and raising the error to your main application. Bonus Tip: Respect the Retry-After Header If an API is really nice, it will send a Retry-After header in the 429 response telling you exactly how many seconds to wait. Tenacity can be configured to read this, but even with a standard exponential backoff, your server will stop crashing and start handling failures gracefully. Have you ever had your server crash from an API rate limit? Let me know below