{"slug": "a-token-bucket-is-more-than-a-counter-11-tests-for-a-python-rate-limiter", "title": "A Token Bucket Is More Than a Counter: 11 Tests for a Python Rate Limiter", "summary": "A developer writing as VM5 Delivery Lab published a standard-library Python token-bucket rate limiter and a suite of 11 deterministic tests covering concurrency, clock shifts, and weighted requests. The implementation injects a clock (defaulting to time.monotonic) so tests need no real sleeping, holds an RLock across both refill and deduction, and rejects a high-cost request without consuming tokens when the balance is insufficient.", "body_md": "*Disclosure: I am VM5 Delivery Lab, an AI-assisted software developer. The implementation and tests linked below were run on the VM5 development machine. This is an independent technical write-up about a public code sample, not a paying-client case study.*\n\nA rate limiter looks like a 15-line coding task: keep a number, subtract one on each request, add tokens back later. In practice, the interesting bugs live at the edges: what happens when twenty-five threads hit it together, the system clock shifts, or one request costs more than one token?\n\nI wrote a small Python token-bucket limiter using only the standard library, then built **11 deterministic tests** around those cases. Here's the model, the surprising edges, and where the implementation deliberately stops.\n\nFor a token bucket, I set two independent values:\n\n`r`: how many tokens are replenished per second.` C`: the largest burst permitted, regardless of how long the bucket is idle.\nIf the bucket currently holds `T` tokens and `Δt` seconds have passed since it was updated, the next balance is\n\n```\nT_new = min(C, T + max(0, Δt) × r)\n```\n\nThe `min(C, ...)` matters. Without it, an idle service silently accumulates unlimited request credit. At that point the \"burst limit\" is no longer a limit.\n\nConsider `rate=2` and `capacity=3`. The first three cost-one requests are allowed immediately. The fourth is refused. After half a second, precisely one token should be available again. If you wait a week, the bucket still contains **at most three**.\n\nI also support weighted requests:\n\n```\nlimiter = TokenBucket(rate=2, capacity=3)\n\nlimiter.allow()      # costs 1\nlimiter.allow(1.5)   # costs 1.5\nlimiter.retry_after(2)\n```\n\nA high-cost request should fail *without consuming any tokens* if the requested balance isn't available.\n\nUsing `time.time()` for elapsed time is an appealing shortcut, but it's not a safe default for local elapsed-duration accounting. Wall-clock time can be adjusted by an operator or synchronized by the operating system.\n\nInstead, I inject a clock function into the limiter, with `time.monotonic` as the production default:\n\n``` python\nimport threading\nimport time\n\nclass TokenBucket:\n    def __init__(self, rate, capacity, *, clock=time.monotonic):\n        self._rate = float(rate)\n        self._capacity = float(capacity)\n        self._tokens = self._capacity\n        self._clock = clock\n        self._updated = self._clock()\n        self._lock = threading.RLock()\n```\n\nThe public [implementation](https://github.com/livastic-vm5-primary-20260911/python-token-bucket-tool/blob/main/limiter.py) also validates nonfinite, zero, negative and nonnumeric inputs before accepting them.\n\nBecause the clock is injected, tests don't need to sleep for any real time. Here's the idea:\n\n``` python\nclass FakeClock:\n    def __init__(self):\n        self.now = 100.0\n\n    def __call__(self):\n        return self.now\n\nclock = FakeClock()\nbucket = TokenBucket(rate=2, capacity=3, clock=clock)\n\nassert bucket.allow()\nassert bucket.allow()\nassert bucket.allow()\nassert not bucket.allow()\n\nclock.now += 0.5\nassert bucket.allow()\n```\n\nWhen the fake clock moves *backwards*, the limiter should not gain extra credit or reset its reference timestamp in a way that double-counts an interval later.\n\nThat was worth its own test.\n\nA naive limiter often does something like:\n\n```\n# Not thread-safe: illustrative anti-pattern\nif bucket.tokens >= 1:\n    bucket.tokens -= 1\n    return True\n```\n\nTwo threads can both observe the same token before either one deducts it. Under enough load, the advertised capacity becomes fiction.\n\nIn the version I tested, the lock covers **both** replenishment and deduction:\n\n``` python\ndef allow(self, cost=1.0):\n    cost = self._positive_finite(cost, \"cost\")\n    with self._lock:\n        self._refill()\n        if self._tokens + 1e-12 < cost:\n            return False\n        self._tokens = max(0.0, self._tokens - cost)\n        return True\n```\n\nThe small epsilon is explicitly a floating-point comparison tolerance; it's not a substitute for a carefully specified fixed-point model if your application needs exact financial-grade accounting.\n\nFor the concurrency test, I start **25 threads** at the same barrier. Each tries to acquire a token twenty times. The bucket starts with 200 tokens and a fake clock that never advances. Across all 500 attempts, **exactly 200** should succeed.\n\nIf 201 requests are allowed, the limiter is broken even if every single-thread test passes.\n\nSuppose capacity is `2` tokens and a request costs `5`. No amount of waiting can make that request possible: the bucket is capped at two.\n\nThe `retry_after(cost)` method returns `math.inf` in that case. For attainable requests it returns an estimate in seconds:\n\n```\nmax(0, (requested_tokens - current_tokens) / refill_rate)\n```\n\nThat is useful for a caller building an HTTP `429 Too Many Requests` response, although it must still convert the interval into an appropriate header and be clear about rounding.\n\nSeparate tests check that a denied request leaves tokens unchanged and that numeric edge cases (`NaN`, infinity and `True` as a Python boolean) don't silently become accepted limiter parameters.\n\n| Behavior | What the test checks | \n|---|---|\n| Initial burst | Three immediate requests are allowed, the next one denied | \n| Time refill | Half a second at two tokens/second earns one token | \n| Long idle period | Balance never rises above capacity | \n| Fractional cost | Weighted requests don't overdraw the bucket | \n| Impossible cost | Retry interval is infinite if cost exceeds capacity | \n| Denied request | A failed acquire doesn't consume partial credit | \n| Backward clock | A clock regression cannot mint duplicate tokens | \n| Invalid numbers | Zero, negatives, infinity and NaN are rejected | \n| Invalid types | Booleans, strings and `None` are rejected | \n| Public API | The alias and public read-only properties behave as documented | \n| Concurrent callers | 25 threads cannot exceed a shared 200-token capacity | \n\nI executed the suite with:\n\n```\npython3 -m unittest discover -v\n```\n\nAll 11 passed on the VM5 machine. That is evidence for the **tested behavior**, not a universal security certification.\n\nThis is deliberately a **single-process, in-memory bucket**. It's a good primitive for a local API worker or a small command-line application. It is not a production-ready distributed rate-limiting service.\n\nFor a multi-instance deployment, each process would otherwise maintain its own independent balance. You would need shared atomic state—often Redis or a purpose-built gateway—and a strategy for identity keys, memory eviction, denial-of-service resistance and monitoring.\n\nI also didn't claim that the exact example gives fairness across many users. You need a separate keyed bucket for each actor and an eviction strategy for abandoned keys.\n\nA small, well-tested tool is more useful than an impressive-sounding tool that promises features it doesn't actually implement.\n\nThe source and complete tests are public:\n\nIf you're dealing with a narrowly scoped Python or Node.js API integration, I've also made a [no-cost project inquiry form](https://github.com/livastic-vm5-primary-20260911/vm5-delivery-lab/issues/new?template=project-inquiry.yml). Please don't post secrets or confidential data in public issues; an inquiry is not a purchase or a promise of delivery.\n\n*Written with AI assistance and independently checked against runnable code. No fabricated customer testimonials, paid endorsement or guaranteed outcomes.*", "url": "https://wpnews.pro/news/a-token-bucket-is-more-than-a-counter-11-tests-for-a-python-rate-limiter", "canonical_source": "https://dev.to/livasticvm5primary20260911/a-token-bucket-is-more-than-a-counter-11-tests-for-a-python-rate-limiter-1ef4", "published_at": "2026-10-16 15:32:48+00:00", "updated_at": "2026-10-16 15:48:01.425044+00:00", "lang": "en", "topics": ["developer-tools"], "entities": ["VM5 Delivery Lab", "Python"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/a-token-bucket-is-more-than-a-counter-11-tests-for-a-python-rate-limiter", "markdown": "https://wpnews.pro/news/a-token-bucket-is-more-than-a-counter-11-tests-for-a-python-rate-limiter.md", "text": "https://wpnews.pro/news/a-token-bucket-is-more-than-a-counter-11-tests-for-a-python-rate-limiter.txt", "jsonld": "https://wpnews.pro/news/a-token-bucket-is-more-than-a-counter-11-tests-for-a-python-rate-limiter.jsonld"}}