I ship a small Cloudflare Worker (a Claude Code status line that pays users a cut of disclosed sponsor revenue -- not the point of this post, just context for where the traffic pattern came from). Every install polls /line every 10-20 seconds while the user is coding. That's a lot of requests hitting one Worker.
Rate limiting a Worker endpoint by install ID looks like a one-liner with KV:
async function checkLineRateLimit(env, installId) {
const key = `ratelimit:${installId}`;
const raw = await env.INSTALLS.get(key);
const count = raw ? parseInt(raw, 10) : 0;
if (count >= LIMIT) return { ok: false };
await env.INSTALLS.put(key, String(count + 1), { expirationTtl: WINDOW_SECONDS });
return { ok: true };
}
This worked in testing. It broke in production, and not because of an attack -- just real usage at real scale.
Workers KV on the free plan has an account-wide cap: 100,000 get() reads per day, not per namespace, not per key. Every single /line call was doing a get() just to check the rate limit, before it ever touched the actual install data. At normal polling frequency across a few hundred concurrent installs, that hot-path read alone was enough to burn through the daily quota -- and once it's gone, it's gone until midnight UTC. Every KV get() anywhere in the account starts failing with a hard error, including reads that have nothing to do with rate limiting.
The failure mode is brutal because it's silent until it isn't: everything works fine right up until the exact moment the account-wide counter ticks over, and then every /line call in production starts 500ing at once.
The rate limit check doesn't need durability. It doesn't need to survive a Worker restart. It just needs to answer "has this install called recently" for a few seconds, cheaply, at massive read volume. That's exactly what the platform's own edge cache is for, and unlike KV, caches.default reads don't count against any daily read quota at all:
async function checkLineRateLimit(env, installId) {
const cache = caches.default;
const cacheKey = new Request(
`https://ratelimit.internal/line/install/${encodeURIComponent(installId)}`
);
try {
const cached = await cache.match(cacheKey);
const count = cached ? parseInt(await cached.text(), 10) : 0;
if (count >= LIMIT) return { ok: false };
await cache.put(
cacheKey,
new Response(String(count + 1), {
headers: { "cache-control": `max-age=${WINDOW_SECONDS}` },
})
);
} catch (e) {
// fail open: a rate-limit check failing should never break the real request
}
return { ok: true };
}
Same interface, same call site, zero KV reads for this path. The try/catch matters as much as the cache swap: a rate limiter is a defensive layer, and defensive layers should fail open, not become a second way for the app to go down.
Fixing the hot path wasn't enough on its own, because the actual install data reads (not the rate limit, the real per-user state) were still hitting KV directly on every call. I moved those onto a Durable Object, one instance per install (idFromName(installId)), so KV is only touched once -- on that instance's true cold start -- and every call after that is served from the DO's own durable storage. The DO mirrors writes back to KV asynchronously so anything else that reads install:<id> directly out of KV (an admin dashboard, a cron job) still sees current data without knowing the DO exists.
The general shape of the fix, not just for this bug: if something is read on every request and doesn't strictly need cross-region strong consistency, it probably shouldn't be sharing a quota with your actual source-of-truth reads. Splitting "is this allowed" from "what's the data" onto different storage tiers means a hot path can run as hot as it wants without threatening to take down everything else sharing that account.