{"slug": "multi-tenant-ai-chat-from-hardcoded-config-to-byok-in-4-steps", "title": "Multi-Tenant AI Chat: From Hardcoded Config to BYOK in 4 Steps", "summary": "A developer describes evolving a multi-tenant AI chat system from hardcoded configuration to a bring-your-own-key (BYOK) model in four stages, driven by real customer requests. The progression moves from a static config object to a database-backed settings table, then to event-sourced configuration for auditability, and finally to tenant-provided API keys. The developer emphasizes that each stage was prompted by a genuine user need, such as self-service prompt editing and the ability to trace configuration changes.", "body_md": "Two tenants, two AI providers, two prompts. Sounds simple, and on day one it is. That's the trap. It stays simple right up until customer number two sends their first \"quick question,\" and eighteen months later you're running a small distributed system to answer it. Here's the honest version of that slide, four stages, each one caused by a real human typing a real request into Slack.\n\nTenant A wants OpenAI. Tenant B wants Claude. Both want their own system prompt. The obvious first version: one config object, one row per tenant. What could possibly go wrong. (Everything. Everything could go wrong. But not yet.)\n\n```\n┌─────────────┐     ┌──────────────────┐     ┌─────────────┐\n│  Request    │ →   │  TENANT_CONFIG   │ →   │  Provider   │\n│  (tenantId) │     │  (hardcoded obj) │     │  SDK call   │\n└─────────────┘     └──────────────────┘     └─────────────┘\njs\nconst TENANT_AI_CONFIG = {\n  tenantA: { provider: 'openai', model: 'gpt-5', prompt: 'You are terse and technical.' },\n  tenantB: { provider: 'anthropic', model: 'claude-sonnet-5', prompt: 'You are friendly. Antworte auf Deutsch.' },\n} as const\n\nasync function handleChat(tenantId: string, userMessage: string) {\n  const config = TENANT_AI_CONFIG[tenantId]\n  const client = config.provider === 'openai' ? openai : anthropic\n  return client.chat(config.model, config.prompt, userMessage)\n}\n```\n\nShips in an afternoon. Two tenants, two rows, demo goes great, everyone claps 👏. Put this moment in a frame, it's the calmest the codebase will ever be.\n\nA week in (a week, we didn't even get a full sprint), tenant B messages: \"can we change the prompt ourselves, without waiting for a deploy?\" Fair ask, they know their users, we don't, and also nobody wants to be the on-call engineer who gets paged to edit a string literal. A hardcoded object can't answer that, it needs a rebuild to change a comma.\n\n```\n┌─────────────┐     ┌──────────────────┐     ┌─────────────┐\n│  Request    │ →   │  tenant_settings │ →   │  Provider   │\n│  (tenantId) │     │  (DB row, admin  │     │  SDK call   │\n│             │     │   editable)      │     │             │\n└─────────────┘     └──────────────────┘     └─────────────┘\n```\n\nConfig moves from a code constant to a table the tenant's own admin UI can write to.\n\n``` js\nasync function getAiConfig(tenantId: string) {\n  const row = await ctx.db.tenantSettings.findOne({ tenantId })\n  return row.aiConfig // { provider, model, prompt }\n}\n```\n\nSame shape as before, different source. `handleChat`\n\ndoesn't change at all, it has no idea any of this happened, which is exactly the point of putting the lookup behind one function.\n\nSelf-service is great until it isn't: tenant B's prompt quietly changed last Tuesday, their bot started answering in pirate-speak for reasons nobody can reconstruct, support gets a ticket, and the honest answer is \"we have no idea, the database doesn't remember either.\" A plain DB row just gets overwritten, the past has no representation, it's Ctrl+Z with no undo history.\n\n```\n┌──────────────┐     ┌────────────────┐     ┌──────────────────┐\n│  Admin edits │ →   │  ConfigChanged │ →   │  current config  │\n│  the prompt  │     │  event (who,   │     │  = fold(events)  │\n│              │     │  when, diff)   │     │                  │\n└──────────────┘     └────────────────┘     └──────────────────┘\n```\n\nThe config becomes event-sourced instead of a mutable row: every change is an event, the current value is a projection over them.\n\n```\nasync function updateAiConfig(tenantId: string, patch: Partial<AiConfig>, actor: string) {\n  await ctx.emit('AiConfigChanged', { tenantId, patch, actor, at: ctx.now() })\n}\n\nasync function getAiConfig(tenantId: string): Promise<AiConfig> {\n  const events = await ctx.db.events.find({ tenantId, type: 'AiConfigChanged' })\n  return events.reduce((cfg, e) => ({ ...cfg, ...e.patch }), DEFAULT_AI_CONFIG)\n}\n```\n\nNow \"who changed it and when\" is a query, not a seance 🔮. `handleChat`\n\nstill hasn't changed, it just calls `getAiConfig`\n\n, blissfully unaware it's now talking to an event log instead of a table.\n\nA bigger tenant shows up, the kind that gets its own Slack channel, with two demands: they want to use *their own* OpenAI key (cost control, their own rate limits, their own finance team breathing down their neck), and they want a hard cap on monthly spend so an over-caffeinated intern's script can't turn into a five-figure invoice.\n\n```\n┌──────────────┐     ┌───────────────────────┐     ┌──────────────┐\n│  Request     │ →   │  config.apiKey?       │ →   │  usage < cap?│\n│              │     │  (BYOK, encrypted)    │     │  → call      │\n│              │     │  else our shared key  │     │  → else 429  │\n└──────────────┘     └───────────────────────┘     └──────────────┘\njs\nasync function callProvider(tenantId: string, userMessage: string) {\n  const config = await getAiConfig(tenantId)\n  const usage = await getMonthlyUsage(tenantId)\n  if (config.usageCap && usage >= config.usageCap) {\n    throw new UsageCapExceeded(tenantId)\n  }\n\n  const apiKey = config.byokApiKey ?? process.env.SHARED_API_KEY // BYOK overrides shared key\n  const client = getClient(config.provider, apiKey)\n  const reply = await client.chat(config.model, config.prompt, userMessage)\n\n  await recordUsage(tenantId, reply.usage.totalTokens)\n  return reply\n}\n```\n\nTwo additive fields on the same config, `byokApiKey`\n\nand `usageCap`\n\n, and one counter check before the call. No new architecture, no rewrite of the first three stages, no \"sorry, we need a full quarter to redesign this.\"\n\nEvery stage kept `getAiConfig(tenantId) → { provider, model, prompt, ... }`\n\nas the seam. Storage changed underneath it four times (constant, DB row, event-sourced projection, projection with encrypted secrets) and the call site never noticed, never cared, never even asked. That's the actual lesson: don't design the multi-tenant AI system upfront, design one seam that can absorb whatever the next Slack message throws at it.\n\nSkipped on purpose: provider fallback, streaming, per-model cost tables. Add those when a tenant actually asks, same as everything above. If a tenant asks for a fifth provider before you've read this sentence, that's not a counterexample, that's Tuesday. 🙃", "url": "https://wpnews.pro/news/multi-tenant-ai-chat-from-hardcoded-config-to-byok-in-4-steps", "canonical_source": "https://dev.to/marc_kumiko/multi-tenant-ai-chat-from-hardcoded-config-to-byok-in-4-steps-51aa", "published_at": "2026-08-03 08:35:50+00:00", "updated_at": "2026-08-03 08:44:23.610296+00:00", "lang": "en", "topics": ["artificial-intelligence", "developer-tools", "ai-infrastructure"], "entities": ["OpenAI", "Anthropic", "Claude", "GPT-5"], "alternates": {"html": "https://wpnews.pro/news/multi-tenant-ai-chat-from-hardcoded-config-to-byok-in-4-steps", "markdown": "https://wpnews.pro/news/multi-tenant-ai-chat-from-hardcoded-config-to-byok-in-4-steps.md", "text": "https://wpnews.pro/news/multi-tenant-ai-chat-from-hardcoded-config-to-byok-in-4-steps.txt", "jsonld": "https://wpnews.pro/news/multi-tenant-ai-chat-from-hardcoded-config-to-byok-in-4-steps.jsonld"}}