Building a support agent that refuses to make things up

An engineer building after-sales support agents for e-commerce shops has developed an architecture that prevents AI hallucination by routing every request to a closed set of intents, each mapped to a typed action with an explicit contract. The system uses a whitelist of facts and a natural-language layer that only phrases responses, ensuring the model cannot generate unverified information. Escalation is treated as a normal outcome with distinct reasons for hand-off to human agents.

Most "AI customer service" demos fall apart the same way. You ask a question the model can't answer from data, and instead of stopping, it produces something plausible. In a chat toy that's a curiosity. In after-sales support it's a promise your company now has to honour — a refund that was never approved, a delivery date that doesn't exist, a return window that isn't your policy. I build after-sales agents for e-commerce shops, and almost all the engineering goes into that one problem: making the agent's honesty a property of the architecture, not of the prompt. Here's how that's actually put together. The naive design is one model, one big prompt, and a pile of documents in a vector store. Ask "where is my order 41822?" and the retrieval layer returns the three chunks that look most like the question. If none of them contain order 41822 — because it's a live database row, not a document — the model still gets a context window full of order-shaped text . It will answer. It will be wrong. The fix isn't a better prompt. It's removing the model's ability to answer that class of question at all. Every request the agent handles is routed to exactly one of a fixed set of intents — order status, delivery delay, return, refund status, exchange, invoice, product question, cancellation. That set is closed. There is no fallback intent that means "answer anyway". Each intent maps to a typed action with an explicit contract: @dataclass frozen=True class OrderStatus: """Reads the order system of record. Never generates a status.""" order ref: str def resolve self, ctx: Ctx - Resolution: order = ctx.commerce.get order self.order ref Shopify / WooCommerce / API if order is None: return Resolution.escalate reason=Reason.NOT FOUND, say="I can't find that order number on this account.", return Resolution.answer template="order status", facts=order.public facts , only whitelisted fields Two things matter here. facts is a whitelist, not the row. public facts returns the carrier, the tracking number, the shipped-at timestamp, the current state. It does not return the margin, the internal notes, the customer's other orders, or the fraud score. The model literally cannot leak a field it was never given. The natural-language layer only phrases. The model receives the resolved facts and a template intent, and its job is to write one or two sentences in the shop's tone of voice. It is not asked what the status is. It is told, and asked to say it nicely. Hallucination has no surface to attach to, because there is no question left open at generation time. Resolution.escalate isn't an error path. It's a normal, expected outcome with its own quality bar — arguably the most important one. class Reason Enum : NOT FOUND = auto no matching record OUT OF SCOPE = auto intent not in the closed set POLICY UNCLEAR = auto rule exists but doesn't cover this case LOW CONFIDENCE = auto intent classification below threshold HUMAN REQUESTED = auto customer asked for a person EMOTIONAL = auto anger / distress detected Each reason produces a different hand-off: a different message to the customer, a different priority in the human queue, and a different summary attached to the ticket so the agent who picks it up doesn't restart from zero. The EMOTIONAL branch is worth calling out. A furious customer is not a retrieval failure — the system may have every fact it needs. It's routed to a human anyway, because "technically resolvable" and "should be resolved by a machine" are different questions. Getting that distinction wrong is how automation projects lose the trust they were meant to build. LOW CONFIDENCE deserves a real threshold, calibrated per shop, not a hard-coded 0.7 . And the threshold should be asymmetric : the cost of a wrong refund is not the cost of an unnecessary escalation. Reading an order is safe. Issuing a refund is not. Those two live behind different gates, and the gate is configuration owned by the merchant, not a prompt: actions: order status: { mode: auto } return label: { mode: auto, max value eur: 80 } refund: { mode: propose } drafts it, a human clicks send cancel order: { mode: propose } address change: { mode: auto, before dispatch only: true } propose mode is what makes the first month of a deployment survivable. The agent does the work and produces the action; a human approves it. You watch the approval rate. When a given action has been approved without edit often enough, that's the evidence to flip it to auto — not a vendor's claim, and not a number in a slide. I run each shop on its own instance, in France, on a French model Mistral https://mistral.ai/ . That reads like a marketing line, so here's the engineering reason. Support conversations are among the most sensitive data a shop holds. Not because of the order numbers — because of what customers write around them. Addresses. Health reasons for a return. Financial difficulty. Complaints about a person. Once that transits through a shared multi-tenant pipeline in another jurisdiction, "where is my data" stops being a question you can answer, and becomes a question you forward to a vendor. A dedicated instance also makes reversibility real rather than contractual. If the merchant leaves, the export is a database and a config file, not a support ticket asking for their data back. The EU AI Act's transparency obligation Article 50 pushes the same way: the customer has to know they're talking to a machine. That's simpler to guarantee when the disclosure is in your own message-composition layer than when it's a setting in someone else's dashboard. Honesty about the trade-off: this design resolves fewer conversations than a model with a free hand. A closed intent set can't handle the long tail. Bounded actions can't improvise. propose mode needs a human in the loop for weeks. That's the trade I'd make every time. The failure mode of the permissive design isn't "slightly worse answers" — it's a confident wrong statement that a real person acts on, in a channel where the shop is legally the one who said it. An agent that says "I'll get a human on this" is a mildly disappointing experience. An agent that invents a refund policy is an incident. I'm Amine, founder of Bynevo Labs — we build sovereign after-sales AI agents for French e-commerce, hosted in France on an open-source stack. Happy to talk architecture in the comments.