"Add rate limiting" sounds like one task. It's actually three separate layers, each defending against a different failure mode, with different trade-offs and different amounts of trust you can place in them. I only understood the distinction properly once I had to pick a strategy for two systems solving different problems: LogicVisor (a public AI tool anyone can hit anonymously) and Titan (a payments platform where the cost of a bad actor is different from the cost of someone burning through free AI credits).
Debouncing a search input, greying out a submit button after the first click, backing off exponentially after a 429. All of this makes an app feel considerate. None of it stops anyone. A malicious actor skips your JavaScript entirely and hits the endpoint directly with curl. Client-side limiting is worth doing (it saves you real traffic and prevents accidental double-submits), but it is not a security control. If it's the only thing standing between your API and abuse, you don't have rate limiting, you have a polite suggestion.
This is where LogicVisor and Titan diverge, because they're not defending against the same thing.
LogicVisor: several checks before a single AI token gets spent
LogicVisor is public. Anyone gets 3 free code reviews with no signup, which means the abuse surface is wide open by design. The submission route runs a stack of checks before it ever calls Gemini or Groq, because every AI call costs real money:
// 1. Check if this exact code has already been reviewed by this model
const cachedReview = await getCachedAIReview(preferred_model.id + "-" + canonicalHash);
if (cachedReview) {
return NextResponse.json({ success: true, data: cachedReview }, { status: 201 });
}
// 2. Enforce the actual rate limit (IP + session based for anon users)
const rateLimitResult = await enforceAIRateLimit(user.id, request);
// 3. Slow down premium users intelligently instead of hard-blocking them
const throttleDelay = await getThrottleDelay(user.id);
if (throttleDelay > 0) {
await new Promise((resolve) => setTimeout(resolve, throttleDelay));
}
// 4. Under heavy load, degrade gracefully instead of rejecting outright
const shouldDegrade = await shouldGracefullyDegrade(user.id, "ai_request");
if (shouldDegrade) {
// return a basic, non-AI response instead of a 429
}
A few things worth naming separately, because they get lumped together under "rate limiting" but aren't the same mechanism:
None of this is a single algorithm out of a textbook. It's several cheap, layered checks, ordered so the most expensive resource (the AI call) is the last thing hit, not the first.
Titan: NestJS Throttler backed by Redis
Titan's rate limiting is intentionally boring by comparison, and that's the correct choice for what it is. It's @nestjs/throttler
wired up as a global guard, with Redis swapped in as the storage backend instead of the default in-memory store:
ThrottlerModule.forRootAsync({
imports: [ConfigModule],
inject: [ConfigService],
useFactory: (configService: ConfigService) => ({
throttlers: [
{
ttl: 60000,
limit: 10,
},
],
storage: new ThrottlerStorageRedisService(
configService.get<string>('REDIS_URL'),
),
}),
}),
// registered alongside the JWT guard as a global APP_GUARD
{
provide: APP_GUARD,
useClass: ThrottlerGuard,
},
10 requests per 60 seconds, globally, enforced before a request even reaches a controller. No per-route tuning yet, no per-user tiers, just a flat ceiling applied everywhere via APP_GUARD
.
The reason Redis matters here isn't exotic: an in-memory counter only knows about requests hitting that one process. The moment you run more than one instance behind a load balancer, in-memory rate limiting stops being true rate limiting, each instance is independently under-counting. Swapping the storage to ThrottlerStorageRedisService
gives every instance a shared, consistent view of who's made how many requests, which is the actual requirement once you're not running a single box.
LogicVisor's layered, identity-aware approach makes sense for a system where the main threat is "someone is farming free AI reviews." Titan's flat, Redis-backed throttle makes sense for a system where the main threat is "someone is hammering an endpoint," and the priority is consistency across instances over nuance per user.
Postgres and Supabase both support rate limiting closer to the data layer (unlogged tables with triggers, PL/pgSQL token bucket functions, TTL-based counters in Mongo). I don't use this in either project, and I don't think I should yet. It's the right call for protecting specific high-value operations (financial writes, heavy analytical queries) where you need the database itself to be the source of truth and can't tolerate a race between the app layer and the DB. LogicVisor's canonical-hash cache lookup is adjacent to this idea (checking the data layer before doing expensive work), but it's a caching pattern, not a rate limiter, and I'd rather keep that distinction honest than dress it up as something it isn't.
Cover photo by HsinKai Tai on Unsplash