# Don't Let Your Support Agent Sign Its Own Permission Slips > Source: > Published: 2026-08-19 12:08:17+00:00 [AI](https://sourcefeed.dev/c/ai)Article # Don't Let Your Support Agent Sign Its Own Permission Slips A circulating reference architecture for workflow-executing agents gets the shape right and the enforcement dangerously wrong. [Priya Nair](https://sourcefeed.dev/u/priya_nair) A reference architecture for customer-service agents that actually *do* things — process the refund, update the account, cut the return label — has been making the rounds, courtesy of a consulting shop called Dextra Labs. The distinction it draws is real and worth internalizing: a RAG chatbot that retrieves policy text and explains it is a search box with manners. An agent that executes the workflow changes database state. That's a different engineering problem, with a different failure surface, and most of the industry's 2024-era "AI support" deployments quietly stayed on the safe side of that line. The architecture itself is a reasonable sketch of current consensus: a loop of intent classification, context retrieval, action planning, tool execution, response generation, and state persistence, running on [Claude](https://www.anthropic.com/claude) Sonnet 4.5 with a handful of typed tools (`lookup_order` , `process_refund` , `create_return_label` , `update_account_field` ). Session state lives in Redis so a customer who verified identity in web chat isn't re-interrogated over email. Escalation fires on hard rules (legal keywords, explicit requests) and soft ones (intent confidence under 0.65, more than eight turns, more than ten minutes). The handoff payload includes sentiment, lifetime value, everything already verified, and a recommended action — a structured brief, not a transcript for a human to reconstruct. If you're building one of these, that sketch is a fine starting point. But two things deserve more scrutiny than the diagram invites. ## The model can't sign its own permission slip The post's most interesting idea is enforcing policy "at the tool signature level": `process_refund` requires a `policy_check_passed: true` field, `update_account_field` requires `verified: true` . The instinct is exactly right — policy belongs at the tool boundary, not in the prompt. Prompts are suggestions; schemas are contracts. But look at who supplies those booleans. If the model fills in `policy_check_passed` as part of its tool call, you've built a system where the entity being constrained attests to its own compliance. An LLM under pressure from a frustrated customer — or a prompt injection buried in an order note — will happily emit `true` , because emitting `true` is what completes the task. The schema gives you a false sense of a guardrail while the actual check lives in the model's judgment, which is the thing you were trying not to trust. The production version of this pattern runs the check server-side: the tool handler receives the refund request, *independently* verifies eligibility against the policy engine and the session's verified-identity record, and rejects the call if it fails. The boolean in the schema can stay — it's a useful forcing function that makes the model reason about policy before acting — but it must be advisory. Treat the LLM as an untrusted planner proposing actions to a deterministic executor that holds the actual authority. [Anthropic's own guidance on building agents](https://www.anthropic.com/engineering/building-effective-agents) points the same direction: invest in the agent-computer interface and keep the deterministic scaffolding doing the load-bearing work. The same logic applies to money movement generally. A refund tool without idempotency keys will double-refund the first time a timeout triggers a retry, and agent loops retry constantly. That's not an AI problem; it's the same distributed-systems hygiene payment engineers have practiced for decades, now mandatory because your caller is a stochastic process. ## Policy compliance is the benchmark, and models still fail it There's independent evidence that this — not orchestration — is where these systems break. Sierra, which sells exactly this category of agent, built [τ-bench](https://github.com/sierra-research/tau2-bench) to measure it: simulated retail and airline customer-service tasks where success requires the final database state to be correct *and* the trajectory to comply with written policy. When the original paper landed in mid-2024, the best function-calling agents completed fewer than half the tasks, and consistency was worse — under 25% pass^8 in retail, meaning an agent that nailed a task once usually couldn't nail it eight times in a row. Frontier models have climbed considerably since, and the τ²-bench successor added a harder telecom domain in response, but the benchmark's core finding stands: the gap between "called the right tool" and "followed the fifty-line refund policy including the edge case about partial shipments" is where agents die. That has a direct practical consequence for evaluation. If you demo your agent completing a refund once, you've measured almost nothing. The metric that matters is pass^k — run the same scenario many times and count how often it succeeds *every* time — because a support agent that's right 90% of the time on refunds is issuing wrong refunds daily at any real volume. Build a simulated-user eval harness before launch, not after the first incident. ## Klarna already ran this experiment for you The business arc here is well documented. [Klarna](https://www.klarna.com) put an AI assistant in front of two-thirds of its support chats in early 2024 — 2.3 million conversations in the first month, the estimated work of 700 full-time agents, roughly $40M in projected annual savings. By May 2025, CEO Sebastian Siemiatkowski was publicly conceding that cost-driven automation had produced lower-quality service and rehiring humans into a hybrid model. Notably, Klarna didn't rip the AI out — it kept expanding automated coverage while restoring a human path for the cases that need one. Read that as validation of the escalation design, not a cautionary tale against agents. The lesson isn't "AI support fails"; it's that the escalation path is a first-class product surface, not an error handler. An agent that knows precisely when to stop — and hands a human a structured brief with verified context and a recommended action instead of a raw transcript — is what separates the Klarna endgame from the Klarna backlash. Teams consistently underinvest here because escalation feels like the failure branch. It's actually where trust is won. ## The verdict The workflow-executing agent is not hype — the tool-use loop, cross-channel state, and tiered escalation described here are legitimately the standard shape of these systems in 2026, and the pieces (function calling, MCP for tool integration, session stores) are commodity. What's still hard, and what no architecture diagram gives you, is the boring perimeter: server-side policy enforcement the model can't talk its way past, idempotent mutations, rate limiting and retries against fragile internal APIs, an audit log that can answer "why did the agent refund this order" six months later, and pass^k evals that treat reliability as the product. Budget most of your engineering time there. The agent loop is a weekend; the perimeter is the project. ## Sources & further reading - [Building a Customer Service AI Agent That Executes Workflows (Not Just Answers Questions) - Full Architecture](https://dev.to/dextralabs/building-a-customer-service-ai-agent-that-executes-workflows-not-just-answers-questions-full-4dd3)— dev.to - [tau2-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domains](https://github.com/sierra-research/tau2-bench)— github.com - [tau-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domains](https://arxiv.org/abs/2406.12045)— arxiv.org - [Klarna changes its AI tune and again recruits humans for customer service](https://www.customerexperiencedive.com/news/klarna-reinvests-human-talent-customer-service-AI-chatbot/747586/)— customerexperiencedive.com - [Building Effective Agents](https://www.anthropic.com/engineering/building-effective-agents)— anthropic.com [Priya Nair](https://sourcefeed.dev/u/priya_nair)· AI & Developer Experience Writer Priya covers AI frameworks, developer productivity tooling, and the startup ecosystem across South and Southeast Asia, bringing a researcher's rigour and a practitioner's empathy to every story. She is deeply sceptical of benchmarks and asks hard questions so her readers don't have to. ## Discussion 0 No comments yet Be the first to weigh in.