Your WhatsApp AI Agent Works in a Demo. Here's What Breaks in Production.

A developer at WAPGROWAI detailed the production-hardening work behind the company's WhatsApp AI agent, separating the cheap classification call that decides whether to answer from the model that writes replies. The agent labels each incoming message as in-scope, out-of-scope, or wants-human via a forced tool call at temperature 0, answers from a single business profile in the system prompt rather than a retrieval index, and rebuilds conversation history on every call.

An AI agent on WhatsApp demos beautifully. You paste in a business description, ask "what time do you open?", and it answers in one friendly sentence. Then real customers arrive. They ask about a competitor's product, send voice notes, type "HUMAN " in capitals, fire three messages in five seconds, and reply to a three-day-old thread. The model provider has a bad afternoon. A customer asks the bot to ignore its instructions. This post goes through what we had to build around the model to run the AI agent in WAPGROWAI https://www.wpautoai.com/ 's WhatsApp inbox, and the questions we think every team should settle before turning one on. The TypeScript snippets are small, simplified versions of the patterns, and each was run against the cases it describes. The most important design decision: the model that writes answers should not be the one deciding whether to answer. Every incoming message first goes to a separate, cheap classification call that returns one of three labels: answer it, decline it, or hand it to a person. The labels come from a forced tool call with an enum, at temperature 0, so the result is never free text you have to parse: // scope.ts export type Scope = "in scope" | "out of scope" | "wants human"; // The model must call this tool; free text is never used as the label. export const LABEL TOOL = { type: "function", function: { name: "label message", parameters: { type: "object", properties: { scope: { type: "string", enum: "in scope", "out of scope", "wants human" } }, required: "scope" , }, }, }; type CallTool = system: string, user: string, tool: typeof LABEL TOOL = Promise<{ scope?: string } ; export const classifyScope = async message: string, businessName: string, callTool: CallTool : Promise<Scope = { const system = Label one customer message for ${businessName}. + "wants human: they ask for a person or are clearly frustrated. " + "out of scope: unrelated to this business. When unsure, choose in scope."; try { const { scope } = await callTool system, message.slice 0, 2000 , LABEL TOOL ; return scope === "out of scope" || scope === "wants human" ? scope : "in scope"; } catch { return "in scope"; // fail open: a classifier outage shouldn't silence the agent } }; export const HANDOFF LINE = "I'll pass you to someone from our team — one moment."; export const refusalLine = businessName: string = Sorry, I can only help with questions about ${businessName}. What can I help you with? ; Two choices in there are deliberate trade-offs: The classifier only sees the current message. That keeps it fast and cheap, and it's why it's told to choose in scope when unsure: a one-word "yes" means nothing without the conversation around it. WAPGROWAI's agent answers from a single business profile the business writes or uploads: what they sell, opening hours, delivery and return rules, prices they're happy to quote, and what the bot must never promise. The whole profile goes into the system prompt on every call, with rules that facts, prices and hours must come from it and that the agent can't place orders, take payments or make bookings. We chose not to use retrieval chunking, embeddings, a vector store for this. A small business's profile is usually a few pages, and the profile is capped at 200,000 characters. Sending it whole avoids an index that drifts out of date and a retrieval step that silently misses the one paragraph that mattered. It costs tokens on every call, which is the trade-off to watch if your knowledge base is large. The model is also told to answer in at most three short sentences and in the customer's language. On WhatsApp, a wall of text is a worse answer than a short one. The model doesn't remember the conversation. You rebuild it on every call from the messages you've stored — and the details matter: // history.ts type StoredMessage = { direction: "in" | "out"; type: string; text?: string }; type Turn = { role: "user" | "assistant"; content: string }; const PLACEHOLDER: Record<string, string = { audio: " sent a voice message ", image: " sent an image ", video: " sent a video ", document: " sent a document ", template: " sent a template message ", }; // Everything the business sent — AI, chatbot or a human agent — is an "assistant" turn. export const buildHistory = messages: StoredMessage , maxTurns = 16, maxChars = 2000 : Turn = messages .map m : Turn | null = { const content = m.text?.trim || PLACEHOLDER m.type ; if content return null; return { role: m.direction === "in" ? "user" : "assistant", content: content.slice 0, maxChars }; } .filter t : t is Turn = t == null .slice -maxTurns ; Customers type in bursts: "hi", "do you deliver to Pune", "and how much". If each message starts its own model call, they finish in random order and the customer gets three overlapping answers. js // lock.ts const inFlight = new Set<string ; // One reply per conversation at a time. A message that arrives mid-reply is skipped, // not queued: the reply being generated already reads the latest history. export const withConversationLock = async <T conversationId: string, job: = Promise<T : Promise<T | null = { if inFlight.has conversationId return null; inFlight.add conversationId ; try { return await job ; } finally { inFlight.delete conversationId ; } }; A message that arrives while a reply is being generated is skipped , not queued — the reply in progress reads the history, so it usually covers it. That's an in-process lock; if you run several instances, you need the same guarantee from something shared a database row or a Redis key . While generating, WAPGROWAI also shows WhatsApp's typing indicator, so a few seconds of thinking doesn't look like silence. The agent calls an OpenAI-compatible endpoint on Cloudflare Workers AI and falls back to NVIDIA's hosted models. What makes failover useful is treating errors differently: // failover.ts export type ErrorClass = "transient" | "quota" | "fatal"; export type Provider = { name: string; call: = Promise<string ; classify: error: unknown = ErrorClass }; const sleep = ms: number = new Promise r = setTimeout r, ms ; // Transient: retry here, then move on. Quota: move on now. Fatal a bad request : stop, // because another provider would reject it too. export const callWithFailover = async providers: Provider , attempts = 2, delayMs = 400 : Promise<string = { let lastError: unknown; for const provider of providers { for let attempt = 1; attempt <= attempts; attempt++ { try { return await provider.call ; } catch error { lastError = error; const kind = provider.classify error ; if kind === "fatal" throw error; if kind === "quota" break; if attempt < attempts await sleep delayMs attempt ; } } } throw lastError; }; Each task gets its own budget, too: the classifier runs at temperature 0 with a short timeout; the answering call gets a little more room; long jobs like generating a chatbot flow get minutes. Models write Markdown. WhatsApp doesn't render headings, and uses single asterisks for bold. Convert before sending, and cap the length at a sentence boundary: js // whatsapp-format.ts const MAX REPLY CHARS = 900; // Models write Markdown; WhatsApp uses bold and has no headings. export const toWhatsApp = reply: string : string = { let text = reply .replace /\ \ .+? \ \ /g, " $1 " .replace / .+? /g, " $1 " .replace /^ {1,6}\s+/gm, "" .trim ; if text.length MAX REPLY CHARS { const cut = text.slice 0, MAX REPLY CHARS ; const lastStop = Math.max cut.lastIndexOf ". " , cut.lastIndexOf "? " , cut.lastIndexOf " " ; text = lastStop 0 ? cut.slice 0, lastStop + 1 : cut; } return text; }; One more filter worth having: some reasoning models occasionally return their chain of thought instead of an answer. Detect that pattern and discard it rather than sending it to a customer. An inbox usually runs several kinds of automation at once. Without a clear order, a keyword rule and the AI agent both reply to the same message. WAPGROWAI's order for each inbound message is: Deterministic flows win over the AI, and the AI wins over everything else. Whatever order you pick, write it down — it's the first thing you'll be asked about when a customer gets two replies. These don't have one right answer, but each needs a deliberate one: The model is the easy part. The production work is everything that decides when it speaks, what it can say, and what happens when it shouldn't. I work on WAPGROWAI https://www.wpautoai.com/ , where this agent runs in the team inbox https://www.wpautoai.com/features/team-inbox . For the non-technical view of what an AI agent should and shouldn't do for a business, see the companion post on Medium. Other posts in this series: webhooks https://dev.to/wapgrowai org/handling-whatsapp-cloud-api-webhooks-in-nodejs-signatures-retries-and-duplicate-events-4dci , the 24-hour window https://dev.to/wapgrowai org/modelling-whatsapps-24-hour-window-correctly-its-harder-than-a-timer-3gho , templates and campaigns https://dev.to/wapgrowai org/from-whatsapp-templates-to-campaigns-what-happens-between-approved-and-delivered-3099 and coexistence https://dev.to/wapgrowai org/whatsapp-coexistence-for-developers-when-the-business-app-and-the-cloud-api-share-a-number-1c4a .