An AI agent on WhatsApp demos beautifully. You paste in a business description, ask "what time do you open?", and it answers in one friendly sentence. Then real customers arrive. They ask about a competitor's product, send voice notes, type "HUMAN!!" in capitals, fire three messages in five seconds, and reply to a three-day-old thread. The model provider has a bad afternoon. A customer asks the bot to ignore its instructions.
This post goes through what we had to build around the model to run the AI agent in WAPGROWAI's WhatsApp inbox, and the questions we think every team should settle before turning one on. The TypeScript snippets are small, simplified versions of the patterns, and each was run against the cases it describes.
The most important design decision: the model that writes answers should not be the one deciding whether to answer.
Every incoming message first goes to a separate, cheap classification call that returns one of three labels: answer it, decline it, or hand it to a person. The labels come from a forced tool call with an enum, at temperature 0, so the result is never free text you have to parse:
// scope.ts
export type Scope = "in_scope" | "out_of_scope" | "wants_human";
// The model must call this tool; free text is never used as the label.
export const LABEL_TOOL = {
type: "function",
function: {
name: "label_message",
parameters: {
type: "object",
properties: { scope: { type: "string", enum: ["in_scope", "out_of_scope", "wants_human"] } },
required: ["scope"],
},
},
};
type CallTool = (system: string, user: string, tool: typeof LABEL_TOOL) => Promise<{ scope?: string }>;
export const classifyScope = async (message: string, businessName: string, callTool: CallTool): Promise<Scope> => {
const system =
`Label one customer message for ${businessName}. ` +
"wants_human: they ask for a person or are clearly frustrated. " +
"out_of_scope: unrelated to this business. When unsure, choose in_scope.";
try {
const { scope } = await callTool(system, message.slice(0, 2000), LABEL_TOOL);
return scope === "out_of_scope" || scope === "wants_human" ? scope : "in_scope";
} catch {
return "in_scope"; // fail open: a classifier outage shouldn't silence the agent
}
};
export const HANDOFF_LINE = "I'll pass you to someone from our team — one moment.";
export const refusalLine = (businessName: string) =>
`Sorry, I can only help with questions about ${businessName}. What can I help you with?`;
Two choices in there are deliberate trade-offs:
The classifier only sees the current message. That keeps it fast and cheap, and it's why it's told to choose in_scope when unsure: a one-word "yes" means nothing without the conversation around it.
WAPGROWAI's agent answers from a single business profile the business writes or uploads: what they sell, opening hours, delivery and return rules, prices they're happy to quote, and what the bot must never promise. The whole profile goes into the system prompt on every call, with rules that facts, prices and hours must come from it and that the agent can't place orders, take payments or make bookings.
We chose not to use retrieval (chunking, embeddings, a vector store) for this. A small business's profile is usually a few pages, and the profile is capped at 200,000 characters. Sending it whole avoids an index that drifts out of date and a retrieval step that silently misses the one paragraph that mattered. It costs tokens on every call, which is the trade-off to watch if your knowledge base is large.
The model is also told to answer in at most three short sentences and in the customer's language. On WhatsApp, a wall of text is a worse answer than a short one.
The model doesn't remember the conversation. You rebuild it on every call from the messages you've stored — and the details matter:
// history.ts
type StoredMessage = { direction: "in" | "out"; type: string; text?: string };
type Turn = { role: "user" | "assistant"; content: string };
const PLACEHOLDER: Record<string, string> = {
audio: "[sent a voice message]",
image: "[sent an image]",
video: "[sent a video]",
document: "[sent a document]",
template: "[sent a template message]",
};
// Everything the business sent — AI, chatbot or a human agent — is an "assistant" turn.
export const buildHistory = (messages: StoredMessage[], maxTurns = 16, maxChars = 2000): Turn[] =>
messages
.map((m): Turn | null => {
const content = m.text?.trim() || PLACEHOLDER[m.type];
if (!content) return null;
return { role: m.direction === "in" ? "user" : "assistant", content: content.slice(0, maxChars) };
})
.filter((t): t is Turn => t !== null)
.slice(-maxTurns);
Customers type in bursts: "hi", "do you deliver to Pune", "and how much". If each message starts its own model call, they finish in random order and the customer gets three overlapping answers.
// lock.ts
const inFlight = new Set<string>();
// One reply per conversation at a time. A message that arrives mid-reply is skipped,
// not queued: the reply being generated already reads the latest history.
export const withConversationLock = async <T>(conversationId: string, job: () => Promise<T>): Promise<T | null> => {
if (inFlight.has(conversationId)) return null;
inFlight.add(conversationId);
try {
return await job();
} finally {
inFlight.delete(conversationId);
}
};
A message that arrives while a reply is being generated is skipped, not queued — the reply in progress reads the history, so it usually covers it. That's an in-process lock; if you run several instances, you need the same guarantee from something shared (a database row or a Redis key). While generating, WAPGROWAI also shows WhatsApp's typing indicator, so a few seconds of thinking doesn't look like silence.
The agent calls an OpenAI-compatible endpoint on Cloudflare Workers AI and falls back to NVIDIA's hosted models. What makes failover useful is treating errors differently:
// failover.ts
export type ErrorClass = "transient" | "quota" | "fatal";
export type Provider = { name: string; call: () => Promise<string>; classify: (error: unknown) => ErrorClass };
const sleep = (ms: number) => new Promise((r) => setTimeout(r, ms));
// Transient: retry here, then move on. Quota: move on now. Fatal (a bad request): stop,
// because another provider would reject it too.
export const callWithFailover = async (providers: Provider[], attempts = 2, delayMs = 400): Promise<string> => {
let lastError: unknown;
for (const provider of providers) {
for (let attempt = 1; attempt <= attempts; attempt++) {
try {
return await provider.call();
} catch (error) {
lastError = error;
const kind = provider.classify(error);
if (kind === "fatal") throw error;
if (kind === "quota") break;
if (attempt < attempts) await sleep(delayMs * attempt);
}
}
}
throw lastError;
};
Each task gets its own budget, too: the classifier runs at temperature 0 with a short timeout; the answering call gets a little more room; long jobs like generating a chatbot flow get minutes.
Models write Markdown. WhatsApp doesn't render headings, and uses single asterisks for bold. Convert before sending, and cap the length at a sentence boundary:
// whatsapp-format.ts
const MAX_REPLY_CHARS = 900;
// Models write Markdown; WhatsApp uses *bold* and has no headings.
export const toWhatsApp = (reply: string): string => {
let text = reply
.replace(/\*\*(.+?)\*\*/g, "*$1*")
.replace(/__(.+?)__/g, "*$1*")
.replace(/^#{1,6}\s+/gm, "")
.trim();
if (text.length > MAX_REPLY_CHARS) {
const cut = text.slice(0, MAX_REPLY_CHARS);
const lastStop = Math.max(cut.lastIndexOf(". "), cut.lastIndexOf("? "), cut.lastIndexOf("! "));
text = lastStop > 0 ? cut.slice(0, lastStop + 1) : cut;
}
return text;
};
One more filter worth having: some reasoning models occasionally return their chain of thought instead of an answer. Detect that pattern and discard it rather than sending it to a customer.
An inbox usually runs several kinds of automation at once. Without a clear order, a keyword rule and the AI agent both reply to the same message. WAPGROWAI's order for each inbound message is:
Deterministic flows win over the AI, and the AI wins over everything else. Whatever order you pick, write it down — it's the first thing you'll be asked about when a customer gets two replies.
These don't have one right answer, but each needs a deliberate one:
The model is the easy part. The production work is everything that decides when it speaks, what it can say, and what happens when it shouldn't.
I work on WAPGROWAI, where this agent runs in the team inbox. For the non-technical view of what an AI agent should and shouldn't do for a business, see the companion post on Medium. Other posts in this series: webhooks, the 24-hour window, templates and campaigns and coexistence.