AIArticle
Injectable time turns an LLM-written rate limiter from trust-me code into a table you can verify in microseconds.
A rate limiter is a nasty thing to review, because reading it tells you almost nothing. The code compiles, the variable names are plausible, the refill math looks right at a glance — and every real defect lives at a boundary you can't see until a client crosses it at 2 a.m. and your support queue fills with 429 complaints. Now multiply that by the way this code increasingly gets written: you ask an LLM for a token bucket, it hands you forty confident lines, and you're the only human who will ever look at them.
A recent Dev.to post by Taylor Wang made a small argument I think deserves a bigger frame: when you review an AI-written rate limiter, the part worth scrutinizing isn't the bucket math. It's whether the code lets you inject a fake clock. That's not a testing nicety. It's the difference between code you can verify and code you have to trust — and with generated code, trust is exactly the thing you don't have.
The seam is the review #
Most generated rate limiters read the wall clock directly — time.time()
, Date.now()
, DateTime.UtcNow
— because that's what the bulk of tutorial code on the internet does, and models reproduce the majority pattern. The moment the limiter reaches for the system clock itself, its behavior becomes a function of when you run it. Your test either sleeps through real seconds (slow, flaky, and still imprecise) or it doesn't test the boundaries at all.
Flip the dependency and everything changes. Pass time in — a clock object, or just an explicit now
argument — and the limiter collapses into a pure function: given this configuration, this key, and this sequence of timestamps, which requests pass? That's a table you can write down before you read a single line of the generated implementation. Which is the right order of operations: start from the failures you want to catch, not from the model's confident prose.
def test_burst_then_refill(limiter):
clock = FakeClock(start=0.0)
rl = limiter(rate=2, burst=3, clock=clock) # 2 req/s, bucket of 3
timeline = [(0.000, True), (0.050, True), (0.100, True), # burst drains
(0.450, False), (0.950, False), # too soon
(1.100, True)] # ~1s: 2 tokens back
for t, expected in timeline:
clock.now = t
assert rl.allow("key-a") is expected, f"at {t}s"
Ten lines, runs in microseconds, and it catches the classic generated-code bugs: refill arithmetic that rounds in the client's favor, integer division silently truncating fractional tokens, and state that's accidentally global — where exhausting key-a
's bucket starts denying key-b
. That last one is worth an explicit test with two keys, because it's invisible in every single-key demo the model was trained on.
If you're prompting for this code, put the seam in the prompt: "accept a clock/now parameter; no direct system time calls." You'll get more testable output, and you've also created a mechanical review rule — grep the diff for direct clock reads and reject on sight. That's a check a reviewer (or CI) can apply without understanding the algorithm, which is precisely what scales when the volume of generated code outstrips human attention.
The ecosystem already voted #
If injectable time sounds like ceremony, notice that platform teams have spent the last few years building it into their foundations. Go 1.25 stabilized testing/synctest, which runs a test in a "bubble" where the
time
package itself is fake and only advances when every goroutine is durably blocked — five-second sleeps complete in microseconds, deterministically. .NET 8 shipped the abstraction with a first-party
TimeProvider
FakeTimeProvider
you Advance()
by hand. Tokio has let Rust tests and auto-advance the runtime clockfor years, and
Jest's fake timersare table stakes in JavaScript.
And the canonical Go limiter, golang.org/x/time/rate, bakes the seam into its public API:
AllowN(now, n)
takes the timestamp as an argument. The maintainers of the reference implementation decided time is an input, not ambient state.That convergence matters for the AI-codegen era more than any of those teams intended. Everyone's review advice for generated code says the same vague thing — "verify behavior, don't trust vibes" — but verification has a cost, and the fake clock is what makes the cost low enough that you'll actually pay it. Deterministic seams turn "review this algorithm" into "check this table," and tables are cheap.
What the fake clock can't tell you #
Be honest about the limits, because a green deterministic suite is its own trap. A single-threaded timeline proves the logic, not the locking — a limiter can pass every boundary test and still corrupt its bucket state under concurrent access, so you still want a race-detector pass or a hammering test with real threads. It also says nothing about capacity or memory growth across a few hundred thousand keys.
And the technique mostly evaporates for distributed limiters. If your bucket lives in Redis and the arithmetic runs in a Lua script against Redis's own TIME
, there's no seam to inject — you're back to integration tests and careful reading. That's a real argument for keeping generated limiters at the edge of what they're good for: single-process, best-effort throttling. For the endpoints where a limiter failure costs money — payments, auth, anything compliance-shaped — use the gateway's limiter or a maintained library, not forty lines from a chat window. On that point the original post and I fully agree.
The transferable lesson is bigger than rate limiting, though. Retry backoff, cache TTLs, token expiry, schedulers — anything an LLM writes that touches time, randomness, or I/O should be reviewed at the seam first. Does the nondeterminism come in as a dependency you control? If yes, twenty minutes buys you a verdict instead of a vibe. If no, send it back before you bother reading the math. The model will happily regenerate; your incident channel is less forgiving.
Sources & further reading #
The Fake Clock Is the Part of an AI-Written Rate Limiter Worth Reviewing— dev.to - Testing Time (and other asynchronicities)— go.dev - Testing with FakeTimeProvider— learn.microsoft.com - tokio::time::— docs.rs - golang.org/x/time/rate— pkg.go.dev
Rachel Goldstein· Dev Tools Editor
Rachel has been embedded in the developer tooling ecosystem for nearly eight years, covering everything from IDE wars and package-manager drama to the quiet rise of AI-assisted coding. She has a soft spot for open-source maintainers and an unhealthy number of terminal emulators installed on a single laptop.
Discussion 0 #
No comments yet
Be the first to weigh in.