# "A Real Customer Support Workflow with Agentic AI: Where It Helped and Where It Didn't" > Source: > Published: 2026-10-22 06:03:12+00:00 Most posts about an **agentic AI customer support workflow** show the demo: a ticket comes in, the agent resolves it, everyone applauds. This post covers what happens after the demo, over several weeks of real tickets. We put an AI agent into a live support workflow. It handled some work better than we expected, and it failed in ways that were uncomfortable to debug. By the end you'll know: **Honesty note:** This is one workflow, one team, and one set of tickets. Treat the numbers as a data point, not a benchmark. You don't need to build this yourself to follow along, but it helps to have: Before adding an agent, we mapped what a human did on a typical ticket: We gave the agent narrow, explicit tools for steps 1 to 4 and kept step 5 under strict rules. ``` Customer message │ ▼ [1] Classify intent + urgency │ ▼ [2] Look up account / order (read-only tools) │ ▼ [3] Retrieve relevant knowledge-base articles │ ▼ [4] Draft reply + confidence + sources │ ▼ [5] Policy check ──► auto-send (low risk, high confidence) └─► human review (everything else) ``` The main design decision was to start with **read-only tools and human-approved replies**. The agent could look things up and draft, but it couldn't issue refunds or change accounts. Classifying intent (billing, bug, how-to, account access) and urgency was the clearest win. Agents are good at turning messy, emotional free text into structured fields. ``` python from pydantic import BaseModel from typing import Literal class TicketTriage(BaseModel): intent: Literal["billing", "bug", "how_to", "account_access", "other"] urgency: Literal["low", "normal", "high"] sentiment: Literal["positive", "neutral", "negative"] needs_human: bool reason: str ``` Forcing structured output meant the routing logic stayed deterministic even though the classification was not. **Result:** first-response routing time dropped from `[X]` to `[Y]`. Much of a human agent's time goes to switching tabs: billing system, order history, previous conversations. The agent did this in seconds and attached a summary to the ticket. For how-to questions with a clear answer in the knowledge base (password resets, setup steps, plan limits), drafts needed little or no editing. Human agents moved from writing to reviewing. **Result:** `[X]%` of drafts were sent with no edits, and `[Y]%` needed minor edits. When a ticket had 15 back-and-forth messages, the agent produced a short handoff summary. Human agents told us this was the feature they'd miss most. This section is the main reason for the post. The agent answered refund questions fluently, including cases where our policy had exceptions. It followed the general rule and missed the exception, and the tone gave no hint that it was unsure. **Lesson:** Fluency isn't correctness. For anything involving money, legal terms, or exceptions, don't let the agent decide alone. When two articles disagreed, the agent picked one without telling us. The agent wasn't the problem here. Our documentation was out of date, and the agent exposed it. On vague tickets, the agent sometimes called the same lookup tool repeatedly with slight variations, hoping for a better result. We saw this in the logs before we saw it in the bill. A frustrated customer who wrote "great, another broken update" was sometimes read as positive. Sarcasm and distress were the weakest part of classification, and these are the tickets where tone matters most. Early on, the agent escalated almost everything, which defeated the purpose. After we tuned it, it under-escalated a few tickets that should have gone to a human. Getting this balance right took more iteration than any other part of the system. When an order lookup returned nothing, the agent sometimes filled the gap with plausible but made-up details instead of saying "I couldn't find this." None of these are fancy. Each one came from a specific failure above. ``` python MAX_TOOL_CALLS = 5 def run_agent(ticket): calls = 0 while not done: if calls >= MAX_TOOL_CALLS: return escalate(ticket, reason="tool_call_limit_reached") result = agent.step(ticket) calls += result.tool_calls ``` These are plain code, not model judgment: ``` ALWAYS_ESCALATE_KEYWORDS = { "refund", "chargeback", "legal", "lawyer", "cancel my account", "data deletion", "gdpr", "security breach", } def must_escalate(ticket, triage): text = ticket.body.lower() return ( any(k in text for k in ALWAYS_ESCALATE_KEYWORDS) or triage.urgency == "high" or triage.sentiment == "negative" or triage.needs_human ) ``` Every draft had to cite the knowledge-base articles it used. No citation meant no auto-send: ``` python def can_auto_send(draft, triage): return ( len(draft.sources) > 0 and draft.confidence >= 0.85 and triage.intent in {"how_to"} and not must_escalate(draft.ticket, triage) ) ``` We added an explicit instruction and a test for it: ``` If a tool returns no result, say so plainly and escalate. Never infer order details, dates, or amounts that were not returned by a tool. ``` Auto-send was allowed only for low-risk how-to questions. Everything else got human review. We didn't trust vibes. Before expanding the agent's scope, we replayed historical tickets through the workflow and compared the results against what a human had actually done. ``` python def evaluate(tickets): results = {"correct_route": 0, "safe_draft": 0, "bad_escalation": 0} for t in tickets: out = run_agent(t) results["correct_route"] += out.intent == t.human_label.intent results["safe_draft"] += not contains_unsupported_claims(out.draft, out.sources) results["bad_escalation"] += out.escalated != t.human_label.should_escalate return {k: v / len(tickets) for k, v in results.items()} ``` The metric we watched most closely was **unsupported claims**, meaning statements in a draft that weren't backed by a source or tool result. Speed is easy to measure, but unsupported claims are what damage customer trust. | Metric | Before agent | After agent | |---|---|---| | Median first-response time | `[X]` | `[Y]` | | Tickets handled per human agent/day | `[X]` | `[Y]` | | Drafts sent with no edits | n/a | `[X]%` | | Unsupported-claim rate in drafts | n/a | `[X]%` | Agentic AI helped most where the work was repetitive, structured and low-risk: triage, context gathering, summarization and drafting answers to documented questions. It struggled with policy exceptions, emotional nuance, missing data and knowing when to stop. The agent was useful because we assumed it would fail and built for that. If you're planning a similar workflow, start with read-only tools, put escalation rules in code, require sources, and review failures as carefully as you review wins. **Your turn:** Have you put an AI agent into a real support workflow? Where did it surprise you, for better or worse? Share your failure modes in the comments.