How to Handle API Rate Limits in Python Like a Pro (with Exponential Backoff)

If you are building applications that rely on external APIs (like OpenAI, Gemini, GitHub, or even just fetching job data), you have probably run into the dreaded 429 Too Many Requests error.

When I was building my AI backend, my server kept crashing because I was hammering the Gemini API too fast.

If you just write a try/except block and instantly try again, you will get banned. You need Exponential Backoff. Here is how it works and how to build it in Python.

What is Exponential Backoff?

Instead of retrying a failed request immediately, you wait a little bit. If it fails again, you wait twice as long. If it fails again, you wait twice as long as that.

· Attempt 1 fails -> Wait 1 second.

· Attempt 2 fails -> Wait 2 seconds.

· Attempt 3 fails -> Wait 4 seconds.

· Attempt 4 fails -> Wait 8 seconds.

This gives the API server time to breathe and clear its queue, making your request much more likely to succeed on the next try.

The Python Code (Using the tenacity Library)

You could write this logic from scratch using a while loop, but why do that when Python has an incredible library called tenacity?

First, install it:

pip install tenacity

Now, let's build a robust API request function:

import httpx
from tenacity import retry, stop_after_attempt, wait_exponential, retry_if_exception_type

def is_rate_limit_error(exception):
    return isinstance(exception, httpx.HTTPStatusError) and exception.response.status_code in [429, 500]

@retry(
    retry=retry_if_exception_type(httpx.HTTPStatusError), 
    wait=wait_exponential(multiplier=1, min=2, max=10),
    stop=stop_after_attempt(5)
)
async def fetch_external_data(url: str):
    print(f"Attempting to fetch: {url}")
    async with httpx.AsyncClient() as client:
        response = await client.get(url, timeout=10.0)

        response.raise_for_status() 
        return response.json()

What does this code do?

· wait_exponential(multiplier=1, min=2, max=10): This is the magic. It will wait 2s, then 4s, then 8s, capping at 10s so you don't wait forever.

· stop_after_attempt(5): It will try 5 times before finally giving up and raising the error to your main application.

Bonus Tip: Respect the Retry-After Header

If an API is really nice, it will send a Retry-After header in the 429 response telling you exactly how many seconds to wait. Tenacity can be configured to read this, but even with a standard exponential backoff, your server will stop crashing and start handling failures gracefully.

Have you ever had your server crash from an API rate limit? Let me know below!