Two tenants, two AI providers, two prompts. Sounds simple, and on day one it is. That's the trap. It stays simple right up until customer number two sends their first "quick question," and eighteen months later you're running a small distributed system to answer it. Here's the honest version of that slide, four stages, each one caused by a real human typing a real request into Slack.
Tenant A wants OpenAI. Tenant B wants Claude. Both want their own system prompt. The obvious first version: one config object, one row per tenant. What could possibly go wrong. (Everything. Everything could go wrong. But not yet.)
┌─────────────┐ ┌──────────────────┐ ┌─────────────┐
│ Request │ → │ TENANT_CONFIG │ → │ Provider │
│ (tenantId) │ │ (hardcoded obj) │ │ SDK call │
└─────────────┘ └──────────────────┘ └─────────────┘
js
const TENANT_AI_CONFIG = {
tenantA: { provider: 'openai', model: 'gpt-5', prompt: 'You are terse and technical.' },
tenantB: { provider: 'anthropic', model: 'claude-sonnet-5', prompt: 'You are friendly. Antworte auf Deutsch.' },
} as const
async function handleChat(tenantId: string, userMessage: string) {
const config = TENANT_AI_CONFIG[tenantId]
const client = config.provider === 'openai' ? openai : anthropic
return client.chat(config.model, config.prompt, userMessage)
}
Ships in an afternoon. Two tenants, two rows, demo goes great, everyone claps 👏. Put this moment in a frame, it's the calmest the codebase will ever be.
A week in (a week, we didn't even get a full sprint), tenant B messages: "can we change the prompt ourselves, without waiting for a deploy?" Fair ask, they know their users, we don't, and also nobody wants to be the on-call engineer who gets paged to edit a string literal. A hardcoded object can't answer that, it needs a rebuild to change a comma.
┌─────────────┐ ┌──────────────────┐ ┌─────────────┐
│ Request │ → │ tenant_settings │ → │ Provider │
│ (tenantId) │ │ (DB row, admin │ │ SDK call │
│ │ │ editable) │ │ │
└─────────────┘ └──────────────────┘ └─────────────┘
Config moves from a code constant to a table the tenant's own admin UI can write to.
async function getAiConfig(tenantId: string) {
const row = await ctx.db.tenantSettings.findOne({ tenantId })
return row.aiConfig // { provider, model, prompt }
}
Same shape as before, different source. handleChat
doesn't change at all, it has no idea any of this happened, which is exactly the point of putting the lookup behind one function.
Self-service is great until it isn't: tenant B's prompt quietly changed last Tuesday, their bot started answering in pirate-speak for reasons nobody can reconstruct, support gets a ticket, and the honest answer is "we have no idea, the database doesn't remember either." A plain DB row just gets overwritten, the past has no representation, it's Ctrl+Z with no undo history.
┌──────────────┐ ┌────────────────┐ ┌──────────────────┐
│ Admin edits │ → │ ConfigChanged │ → │ current config │
│ the prompt │ │ event (who, │ │ = fold(events) │
│ │ │ when, diff) │ │ │
└──────────────┘ └────────────────┘ └──────────────────┘
The config becomes event-sourced instead of a mutable row: every change is an event, the current value is a projection over them.
async function updateAiConfig(tenantId: string, patch: Partial<AiConfig>, actor: string) {
await ctx.emit('AiConfigChanged', { tenantId, patch, actor, at: ctx.now() })
}
async function getAiConfig(tenantId: string): Promise<AiConfig> {
const events = await ctx.db.events.find({ tenantId, type: 'AiConfigChanged' })
return events.reduce((cfg, e) => ({ ...cfg, ...e.patch }), DEFAULT_AI_CONFIG)
}
Now "who changed it and when" is a query, not a seance 🔮. handleChat
still hasn't changed, it just calls getAiConfig
, blissfully unaware it's now talking to an event log instead of a table.
A bigger tenant shows up, the kind that gets its own Slack channel, with two demands: they want to use their own OpenAI key (cost control, their own rate limits, their own finance team breathing down their neck), and they want a hard cap on monthly spend so an over-caffeinated intern's script can't turn into a five-figure invoice.
┌──────────────┐ ┌───────────────────────┐ ┌──────────────┐
│ Request │ → │ config.apiKey? │ → │ usage < cap?│
│ │ │ (BYOK, encrypted) │ │ → call │
│ │ │ else our shared key │ │ → else 429 │
└──────────────┘ └───────────────────────┘ └──────────────┘
js
async function callProvider(tenantId: string, userMessage: string) {
const config = await getAiConfig(tenantId)
const usage = await getMonthlyUsage(tenantId)
if (config.usageCap && usage >= config.usageCap) {
throw new UsageCapExceeded(tenantId)
}
const apiKey = config.byokApiKey ?? process.env.SHARED_API_KEY // BYOK overrides shared key
const client = getClient(config.provider, apiKey)
const reply = await client.chat(config.model, config.prompt, userMessage)
await recordUsage(tenantId, reply.usage.totalTokens)
return reply
}
Two additive fields on the same config, byokApiKey
and usageCap
, and one counter check before the call. No new architecture, no rewrite of the first three stages, no "sorry, we need a full quarter to redesign this."
Every stage kept getAiConfig(tenantId) → { provider, model, prompt, ... }
as the seam. Storage changed underneath it four times (constant, DB row, event-sourced projection, projection with encrypted secrets) and the call site never noticed, never cared, never even asked. That's the actual lesson: don't design the multi-tenant AI system upfront, design one seam that can absorb whatever the next Slack message throws at it.
Skipped on purpose: provider fallback, streaming, per-model cost tables. Add those when a tenant actually asks, same as everything above. If a tenant asks for a fifth provider before you've read this sentence, that's not a counterexample, that's Tuesday. 🙃