AIArticle Anthropic workspaces already enforce per-tenant limits; your dependency injection should do the wiring, not the throttling.
Priya Nair
A pattern has been making the rounds for multi-tenant Claude apps: use FastAPI's dependency injection to resolve a tenant from the JWT, pull their encrypted API key from the database, hand each request a tenant-scoped Anthropic
client, and gate it behind an in-process rate-limit bucket. The problem it names is real — one write-up describes hitting it at 15 tenants, when a single customer's agentic loop on a shared key could throttle everyone else. The wiring is genuinely good FastAPI. The enforcement strategy, though, is building in the application layer what Anthropic already ships in the platform layer, and the DIY version fails in ways that only show up in production.
The noisy-neighbor problem is real — and it's org-shaped #
Anthropic enforces rate limits at the organization level: requests per minute, input tokens per minute, and output tokens per minute, per model, replenished by a token bucket. If all your tenants ride one API key, they share one bucket. When tenant A burns through your OTPM, tenant B gets the 429 — and from your telemetry, you can't even tell whose traffic did it. That's the ticking time bomb, and it has nothing to do with FastAPI. Any framework, any language, same failure.
So the instinct to isolate per tenant is correct. The question is where the isolation should live.
What the DIY version gets right, and where it breaks #
Credit where due: injecting a per-tenant client through Depends()
instead of a module-level singleton is the right shape, and the pattern's explicit refusal to slap @lru_cache
on the tenant lookup — because a rotated key would be served stale all day — is a mistake I've watched teams make more than once.
But the enforcement half doesn't survive contact with deployment:
An in-memory sliding window dies at Every Uvicorn worker gets its own bucket, so a "60 requests per minute" limit quietly becomes 60 × N. The honest fix is--workers 2
.Redis, which means you're now operating a distributed rate limiter — the exact infrastructure a gateway would have given you.It counts the wrong unit. Anthropic's binding limits are token-denominated (ITPM/OTPM), not request-denominated. Sixty small classification calls and sixty 100K-token agent turns are wildly different loads; a request counter treats them identically. The API tells you the truth in itsanthropic-ratelimit-*
response headers — remaining tokens, reset times,retry-after
— and a client-side counter that ignores them is guessing.Tutorial code rots. The circulating example pinsclaude-3-5-sonnet-20241022
, a model Anthropic retired in October 2025. Anyone pasting it today gets a 404 before they get a rate limit.
The platform already does this #
Here's the part the DIY discussion skips entirely: Anthropic Console workspaces are per-tenant isolation as a managed feature. Each workspace gets its own API keys, and you can cap it below your org limits on exactly the axes that matter — RPM, ITPM, OTPM per model tier, plus a monthly spend limit with alert thresholds. Workspaces are creatable programmatically through the Admin API, every response carries an anthropic-workspace-id
header so attribution is free, and the Usage and Cost API breaks down spend by workspace — which turns your tenant billing report from a log-scraping project into one API call.
Map one tenant to one workspace and the enforcement problem mostly evaporates. Tenant A's runaway loop hits their workspace ceiling and their requests 429; everyone else's buckets are untouched. No Redis, no sliding windows, no drift between what your limiter thinks and what Anthropic enforces. Prompt caches are workspace-isolated too, and since cached reads don't count toward ITPM on current models, a tenant with a stable system prompt effectively multiplies their own throughput without eating anyone else's.
The cap is the catch: 100 workspaces per organization. For B2B SaaS at 15, 40, even 80 enterprise tenants — precisely the scale where this pattern gets written up — that's comfortable. For PLG products with thousands of self-serve customers, it's a non-starter, and that's where the middle layer earns its keep.
Above 100 tenants: put a gateway in front #
An LLM gateway like LiteLLM exists for exactly this shape: mint a virtual key per tenant (or a team per tenant), attach rpm_limit
, tpm_limit
, and max_budget
to each, and let the proxy do distributed enforcement and spend tracking against a single upstream Anthropic key. You get per-tenant model allowlists as a bonus — Haiku for the free tier, Opus for enterprise — without touching application code. It's one more piece of infrastructure to run, but it's shared infrastructure with a community hardening it, versus a bucket class you maintain alone.
Where that leaves the FastAPI pattern #
Keep the dependency injection — for wiring, not enforcement. get_tenant()
resolving identity, a Depends()
chain handing your route the right client pointed at the right workspace key or gateway virtual key: that's clean, testable, and correct. What should come out of the app layer is the rate-limit bucket itself. Your process is the worst place in the stack to enforce quotas: it's replicated, it's restartable, and it can't see what Anthropic's actual bucket contains.
The adoption path I'd actually recommend: under ~100 tenants, one workspace per tenant, key stored per tenant, limits set in the Console or Admin API, billing pulled from the Usage and Cost API. Past that, or multi-provider, put LiteLLM (or equivalent) in front and mint virtual keys. In both cases, handle 429s by reading retry-after
instead of pre-guessing, and keep a soft in-app throttle only where you want a friendlier UX than a raw 429 — as a courtesy, not a control.
The tutorial-shaped version of this pattern is how a lot of teams discover the problem. Just don't let it be where you stop: the interesting engineering here isn't the sliding window, it's realizing you never had to build one.
Sources & further reading #
FastAPI Dependency Injection for Anthropic Claude: Isolating API Keys and Rate Limits Per Tenant— dev.to -
[Rate limits](https://platform.claude.com/docs/en/api/rate-limits)— platform.claude.com -
[Workspaces](https://platform.claude.com/docs/en/manage-claude/workspaces)— platform.claude.com -
[LiteLLM Proxy Virtual Keys](https://docs.litellm.ai/docs/proxy/virtual_keys)— docs.litellm.ai
[Priya Nair](https://sourcefeed.dev/u/priya_nair)· AI & Developer Experience Writer
Priya covers AI frameworks, developer productivity tooling, and the startup ecosystem across South and Southeast Asia, bringing a researcher's rigour and a practitioner's empathy to every story. She is deeply sceptical of benchmarks and asks hard questions so her readers don't have to.
Discussion 0 #
No comments yet
Be the first to weigh in.