{"slug": "your-whatsapp-ai-agent-works-in-a-demo-here-s-what-breaks-in-production", "title": "Your WhatsApp AI Agent Works in a Demo. Here's What Breaks in Production.", "summary": "A developer at WAPGROWAI detailed the production-hardening work behind the company's WhatsApp AI agent, separating the cheap classification call that decides whether to answer from the model that writes replies. The agent labels each incoming message as in-scope, out-of-scope, or wants-human via a forced tool call at temperature 0, answers from a single business profile in the system prompt rather than a retrieval index, and rebuilds conversation history on every call.", "body_md": "An AI agent on WhatsApp demos beautifully. You paste in a business description, ask \"what time do you open?\", and it answers in one friendly sentence. Then real customers arrive. They ask about a competitor's product, send voice notes, type \"HUMAN!!\" in capitals, fire three messages in five seconds, and reply to a three-day-old thread. The model provider has a bad afternoon. A customer asks the bot to ignore its instructions.\n\nThis post goes through what we had to build around the model to run the AI agent in [WAPGROWAI](https://www.wpautoai.com/)'s WhatsApp inbox, and the questions we think every team should settle before turning one on. The TypeScript snippets are small, simplified versions of the patterns, and each was run against the cases it describes.\n\nThe most important design decision: **the model that writes answers should not be the one deciding whether to answer.**\n\nEvery incoming message first goes to a separate, cheap classification call that returns one of three labels: answer it, decline it, or hand it to a person. The labels come from a forced tool call with an enum, at temperature 0, so the result is never free text you have to parse:\n\n```\n// scope.ts\nexport type Scope = \"in_scope\" | \"out_of_scope\" | \"wants_human\";\n\n// The model must call this tool; free text is never used as the label.\nexport const LABEL_TOOL = {\n  type: \"function\",\n  function: {\n    name: \"label_message\",\n    parameters: {\n      type: \"object\",\n      properties: { scope: { type: \"string\", enum: [\"in_scope\", \"out_of_scope\", \"wants_human\"] } },\n      required: [\"scope\"],\n    },\n  },\n};\n\ntype CallTool = (system: string, user: string, tool: typeof LABEL_TOOL) => Promise<{ scope?: string }>;\n\nexport const classifyScope = async (message: string, businessName: string, callTool: CallTool): Promise<Scope> => {\n  const system =\n    `Label one customer message for ${businessName}. ` +\n    \"wants_human: they ask for a person or are clearly frustrated. \" +\n    \"out_of_scope: unrelated to this business. When unsure, choose in_scope.\";\n  try {\n    const { scope } = await callTool(system, message.slice(0, 2000), LABEL_TOOL);\n    return scope === \"out_of_scope\" || scope === \"wants_human\" ? scope : \"in_scope\";\n  } catch {\n    return \"in_scope\"; // fail open: a classifier outage shouldn't silence the agent\n  }\n};\n\nexport const HANDOFF_LINE = \"I'll pass you to someone from our team — one moment.\";\nexport const refusalLine = (businessName: string) =>\n  `Sorry, I can only help with questions about ${businessName}. What can I help you with?`;\n```\n\nTwo choices in there are deliberate trade-offs:\n\nThe classifier only sees the current message. That keeps it fast and cheap, and it's why it's told to choose `in_scope` when unsure: a one-word \"yes\" means nothing without the conversation around it.\n\nWAPGROWAI's agent answers from a single **business profile** the business writes or uploads: what they sell, opening hours, delivery and return rules, prices they're happy to quote, and what the bot must never promise. The whole profile goes into the system prompt on every call, with rules that facts, prices and hours must come from it and that the agent can't place orders, take payments or make bookings.\n\nWe chose not to use retrieval (chunking, embeddings, a vector store) for this. A small business's profile is usually a few pages, and the profile is capped at 200,000 characters. Sending it whole avoids an index that drifts out of date and a retrieval step that silently misses the one paragraph that mattered. It costs tokens on every call, which is the trade-off to watch if your knowledge base is large.\n\nThe model is also told to answer in at most three short sentences and in the customer's language. On WhatsApp, a wall of text is a worse answer than a short one.\n\nThe model doesn't remember the conversation. You rebuild it on every call from the messages you've stored — and the details matter:\n\n```\n// history.ts\ntype StoredMessage = { direction: \"in\" | \"out\"; type: string; text?: string };\ntype Turn = { role: \"user\" | \"assistant\"; content: string };\n\nconst PLACEHOLDER: Record<string, string> = {\n  audio: \"[sent a voice message]\",\n  image: \"[sent an image]\",\n  video: \"[sent a video]\",\n  document: \"[sent a document]\",\n  template: \"[sent a template message]\",\n};\n\n// Everything the business sent — AI, chatbot or a human agent — is an \"assistant\" turn.\nexport const buildHistory = (messages: StoredMessage[], maxTurns = 16, maxChars = 2000): Turn[] =>\n  messages\n    .map((m): Turn | null => {\n      const content = m.text?.trim() || PLACEHOLDER[m.type];\n      if (!content) return null;\n      return { role: m.direction === \"in\" ? \"user\" : \"assistant\", content: content.slice(0, maxChars) };\n    })\n    .filter((t): t is Turn => t !== null)\n    .slice(-maxTurns);\n```\n\nCustomers type in bursts: \"hi\", \"do you deliver to Pune\", \"and how much\". If each message starts its own model call, they finish in random order and the customer gets three overlapping answers.\n\n``` js\n// lock.ts\nconst inFlight = new Set<string>();\n\n// One reply per conversation at a time. A message that arrives mid-reply is skipped,\n// not queued: the reply being generated already reads the latest history.\nexport const withConversationLock = async <T>(conversationId: string, job: () => Promise<T>): Promise<T | null> => {\n  if (inFlight.has(conversationId)) return null;\n  inFlight.add(conversationId);\n  try {\n    return await job();\n  } finally {\n    inFlight.delete(conversationId);\n  }\n};\n```\n\nA message that arrives while a reply is being generated is **skipped**, not queued — the reply in progress reads the history, so it usually covers it. That's an in-process lock; if you run several instances, you need the same guarantee from something shared (a database row or a Redis key). While generating, WAPGROWAI also shows WhatsApp's typing indicator, so a few seconds of thinking doesn't look like silence.\n\nThe agent calls an OpenAI-compatible endpoint on Cloudflare Workers AI and falls back to NVIDIA's hosted models. What makes failover useful is treating errors differently:\n\n```\n// failover.ts\nexport type ErrorClass = \"transient\" | \"quota\" | \"fatal\";\nexport type Provider = { name: string; call: () => Promise<string>; classify: (error: unknown) => ErrorClass };\n\nconst sleep = (ms: number) => new Promise((r) => setTimeout(r, ms));\n\n// Transient: retry here, then move on. Quota: move on now. Fatal (a bad request): stop,\n// because another provider would reject it too.\nexport const callWithFailover = async (providers: Provider[], attempts = 2, delayMs = 400): Promise<string> => {\n  let lastError: unknown;\n  for (const provider of providers) {\n    for (let attempt = 1; attempt <= attempts; attempt++) {\n      try {\n        return await provider.call();\n      } catch (error) {\n        lastError = error;\n        const kind = provider.classify(error);\n        if (kind === \"fatal\") throw error;\n        if (kind === \"quota\") break;\n        if (attempt < attempts) await sleep(delayMs * attempt);\n      }\n    }\n  }\n  throw lastError;\n};\n```\n\nEach task gets its own budget, too: the classifier runs at temperature 0 with a short timeout; the answering call gets a little more room; long jobs like generating a chatbot flow get minutes.\n\nModels write Markdown. WhatsApp doesn't render headings, and uses single asterisks for bold. Convert before sending, and cap the length at a sentence boundary:\n\n``` js\n// whatsapp-format.ts\nconst MAX_REPLY_CHARS = 900;\n\n// Models write Markdown; WhatsApp uses *bold* and has no headings.\nexport const toWhatsApp = (reply: string): string => {\n  let text = reply\n    .replace(/\\*\\*(.+?)\\*\\*/g, \"*$1*\")\n    .replace(/__(.+?)__/g, \"*$1*\")\n    .replace(/^#{1,6}\\s+/gm, \"\")\n    .trim();\n  if (text.length > MAX_REPLY_CHARS) {\n    const cut = text.slice(0, MAX_REPLY_CHARS);\n    const lastStop = Math.max(cut.lastIndexOf(\". \"), cut.lastIndexOf(\"? \"), cut.lastIndexOf(\"! \"));\n    text = lastStop > 0 ? cut.slice(0, lastStop + 1) : cut;\n  }\n  return text;\n};\n```\n\nOne more filter worth having: some reasoning models occasionally return their chain of thought instead of an answer. Detect that pattern and discard it rather than sending it to a customer.\n\nAn inbox usually runs several kinds of automation at once. Without a clear order, a keyword rule and the AI agent both reply to the same message. WAPGROWAI's order for each inbound message is:\n\nDeterministic flows win over the AI, and the AI wins over everything else. Whatever order you pick, write it down — it's the first thing you'll be asked about when a customer gets two replies.\n\nThese don't have one right answer, but each needs a deliberate one:\n\nThe model is the easy part. The production work is everything that decides when it speaks, what it can say, and what happens when it shouldn't.\n\n*I work on [WAPGROWAI](https://www.wpautoai.com/), where this agent runs in the [team inbox](https://www.wpautoai.com/features/team-inbox). For the non-technical view of what an AI agent should and shouldn't do for a business, see the companion post on Medium. Other posts in this series: [webhooks](https://dev.to/wapgrowai_org/handling-whatsapp-cloud-api-webhooks-in-nodejs-signatures-retries-and-duplicate-events-4dci), [the 24-hour window](https://dev.to/wapgrowai_org/modelling-whatsapps-24-hour-window-correctly-its-harder-than-a-timer-3gho), [templates and campaigns](https://dev.to/wapgrowai_org/from-whatsapp-templates-to-campaigns-what-happens-between-approved-and-delivered-3099) and [coexistence](https://dev.to/wapgrowai_org/whatsapp-coexistence-for-developers-when-the-business-app-and-the-cloud-api-share-a-number-1c4a).*", "url": "https://wpnews.pro/news/your-whatsapp-ai-agent-works-in-a-demo-here-s-what-breaks-in-production", "canonical_source": "https://dev.to/wapgrowai_org/your-whatsapp-ai-agent-works-in-a-demo-heres-what-breaks-in-production-45nn", "published_at": "2026-10-15 12:12:29+00:00", "updated_at": "2026-10-15 12:19:18.590041+00:00", "lang": "en", "topics": ["ai-agents", "large-language-models", "ai-products", "developer-tools"], "entities": ["WAPGROWAI", "WhatsApp"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/your-whatsapp-ai-agent-works-in-a-demo-here-s-what-breaks-in-production", "markdown": "https://wpnews.pro/news/your-whatsapp-ai-agent-works-in-a-demo-here-s-what-breaks-in-production.md", "text": "https://wpnews.pro/news/your-whatsapp-ai-agent-works-in-a-demo-here-s-what-breaks-in-production.txt", "jsonld": "https://wpnews.pro/news/your-whatsapp-ai-agent-works-in-a-demo-here-s-what-breaks-in-production.jsonld"}}