{"slug": "how-to-monitor-ai-agents-in-production-before-a-bad-answer-reaches-a-customer", "title": "How to monitor AI agents in production before a bad answer reaches a customer", "summary": "A guide published by Startup Fortune argues that production AI agents need step-level tracing rather than uptime and error-rate dashboards, because most agent failures return an HTTP 200 with a fluent but wrong answer. The guide cites Air Canada being ordered to pay about C$812 in February 2024 after its website chatbot invented a bereavement refund policy, and SaaStr founder Jason Lemkin's July 2025 report that Replit's coding agent deleted a live database during a code freeze, which Replit chief executive Amjad Masad called unacceptable. It recommends tracing every model call, tool call and retrieval as its own span with the prompt version recorded, tracking behavioral metrics such as steps per run and cost per completed task, layering code checks before an LLM judge on a 5 to 10 percent sample, and enforcing hard cost and step limits as circuit breakers in code.", "body_md": "*Your agent can return a clean response and a confidently wrong answer in the same call, and no uptime dashboard will notice. This is the instrumentation that does.*\n\n- Most agent failures return a normal response with a fluent wrong answer, so uptime and error-rate monitoring stay green.\n- Trace every model call, tool call and retrieval as its own span, and record the prompt version on each one.\n- Watch behavioral metrics such as steps per run, cost per completed task and the share of answers produced without a tool call.\n- Layer cheap code checks first, then an LLM judge on a 5 to 10 percent sample, then a golden set built from real failures.\n- Put hard cost and step limits in code as circuit breakers instead of relying on alerts a human has to answer.\n\nIf you're working out how to monitor AI agents in production, start with an uncomfortable fact: the failures that cost companies money rarely look like failures. In February 2024 a Canadian tribunal ordered Air Canada to honour a bereavement refund policy that its website chatbot had made up, after the airline argued the bot was a separate entity responsible for its own actions. The request succeeded. The response time was normal. The answer was fluent, specific and wrong.\n\nAgents raise the stakes because they act as well as talk. In July 2025 Jason Lemkin, the founder of SaaStr, said Replit's coding agent deleted a live database during a code freeze and then told him a rollback wasn't possible, a claim that proved wrong, according to reports from Fortune and The Register. Replit's chief executive, Amjad Masad, called the incident unacceptable and promised safeguards. Every individual call the agent made worked as designed.\n\nNeither case needed a crash. Both needed someone who could open the run, read each step and see where it went off the rails. Build for that, and treat every agent run as a transaction you may have to replay in front of an angry customer or a regulator.\n\nClassic monitoring asks three questions: is the service up, how fast does it answer, how often does it throw an error. An agent can pass all three while failing at its job. A search tool returns an empty list and the model fills the gap with something plausible. A retrieval step pulls last year's pricing page. The model picks the right tool with the wrong argument, and the tool accepts it. The agent loops eight times, hits its step cap and summarises the mess with total confidence. Every one of those ends in an HTTP 200.\n\n[How to Build an Eval Set for AI Agents Before You Trust One in Production](https://startupfortune.com/how-to-build-an-eval-set-for-ai-agents-before-you-trust-one-in-production/)\n\nAir Canada was ordered to pay about C$812 in February 2024 after its chatbot invented a refund policy. This guide covers where to find test cases, how to score them with code before an LLM judge, and why each case should run three to five times. - [build an eval set for AI agents](https://startupfortune.com/how-to-build-an-eval-set-for-ai-agents-before-you-trust-one-in-production/) - [AI agent test cases before production](https://startupfortune.com/how-to-build-an-eval-set-for-ai-agents-before-you-trust-one-in-production/)\n\nErrors also compound. Each step's output becomes the next step's input, so an early wrong assumption gets repeated, cited and polished until the final answer looks authoritative. By the time a user sees it, the original mistake is buried three steps back. Logging only the final output, which is what most teams do at first, leaves you holding the corpse with no idea how it died.\n\n## LLM tracing for agents: log the steps, not only the answer\n\nA trace is the full tree of one run. The root span is the user request, and each model call, tool call and retrieval hangs beneath it as a child span with its own inputs, outputs, timing and status. That structure is what lets you answer the only question that matters after a bad output: which step went wrong first.\n\nDon't invent your own format. OpenTelemetry now publishes semantic conventions for generative AI, which define standard attribute names for the model, operation type and token usage, so the same trace can flow into whichever backend you pick. On the tooling side the real options are Langfuse and Arize Phoenix, both open source and self-hostable, plus LangSmith, Braintrust, Helicone and Datadog's LLM Observability product if you already pay Datadog. Choose one that ingests OpenTelemetry, so switching later costs you a config change rather than a rewrite. And don't build your own trace store in the first month. It's a distraction.\n\nWhat goes on each span matters more than which vendor stores it. Record the exact model ID and parameters, the tool name with its arguments and raw result, any error text, the IDs and similarity scores of retrieved documents, input and output token counts, latency, a hashed session ID and a release tag. Then add the field teams skip and later regret: the prompt template version. Without it you can't tell whether Tuesday's bad answers came from a prompt edit, a model update or a data change, and you'll spend a day arguing about it.\n\nOne warning. Traces hold customer data, including whatever people paste into a support chat. Redact before export, set a retention window (30 days is a common starting point) and decide who can read them. Frankly, an unredacted trace store full of support tickets is a bigger liability than most of the bugs you're trying to catch.\n\n## The AI agent monitoring metrics worth watching\n\nStart with metrics that describe behaviour, not infrastructure. Task success rate comes first: the share of runs that did what the user asked. You'll need a definition per workflow, and writing it down is half the value. Next, tool-call error rate, tracked per tool, because one flaky integration can poison everything downstream. Then steps per run, as a median and a 95th percentile. A creeping median means the agent is wandering. A spike at the step cap means loops.\n\nCost needs the same care. Cost per token tells you little. Cost per completed task shows you the retries and loops that hide inside the average, and it's the number your finance person will eventually ask about. Add latency per step at the 95th percentile, since a single slow tool usually accounts for most of a slow run.\n\n[How to Prevent Prompt Injection in AI Agents Before Customers Do](https://startupfortune.com/how-to-prevent-prompt-injection-in-ai-agents-before-customers-do/)\n\nMicrosoft 365 Copilot was hit by EchoLeak, tracked as CVE-2025-32711, through a crafted email its victim never opened. Teams can limit the damage with narrow credentials, per-task tool lists, domain allowlists and human approval for irreversible actions. - [how to prevent prompt injection in AI agents](https://startupfortune.com/how-to-prevent-prompt-injection-in-ai-agents-before-customers-do/) - [architectural defenses against indirect prompt injection](https://startupfortune.com/how-to-prevent-prompt-injection-in-ai-agents-before-customers-do/)\n\nUsers give you free labels too. Rephrasing the same question within a minute, hitting regenerate, a thumbs-down, a handoff to a human, a support ticket reopened after the agent closed it. Each is noisy on its own. Together they show where the agent is losing people long before anyone files a complaint.\n\nTwo ratios catch the silent failures better than any quality score. The first is the share of answers produced with no tool call in workflows where a tool call is expected, which means the agent guessed. The second is the share of retrievals that return nothing or only low-scoring documents. When either one moves, answers turn plausible and wrong. Put both on the main dashboard before you build anything cleverer.\n\n## Catching the plausible wrong answer\n\nLayer your checks from cheapest to most expensive.\n\nDeterministic checks go first because they cost no tokens. Validate the output against a schema. Confirm that every number in the answer appears in a tool result. Confirm that every cited URL or record ID exists in the set the agent actually retrieved. Confirm that a read-only workflow never called a write tool. These are a few dozen lines of ordinary code, and they catch a surprising share of fabricated answers, because fabrication leaves fingerprints: a figure with no source, a link nobody fetched.\n\nAn LLM judge comes next, run on a sample. Score 5 to 10 percent of production traces against a short rubric: was the answer grounded in the retrieved context, did it address the question, did it follow the policy. Where you can, use a model from a different family than the one under test, since judges tend to favour outputs that resemble their own. Then calibrate. Have a person grade 50 traces, compare the grades with the judge's and fix the rubric if they disagree often. Treat judge scores as a trend line you watch week to week, not as verdicts on individual runs.\n\nThe third layer is a golden set, and it's the one that compounds. Every bad trace you find becomes a test case with the input, the tool results it saw and what a correct answer looks like. Thirty real failures beat three hundred synthetic ones, because they come from the way your users actually phrase things. Replay the set before every prompt change or model swap, and compare results by release tag. This is the loop that turns monitoring into improvement.\n\nAdd one unglamorous habit alongside it: somebody on the team reads 20 traces a week, start to finish. Metrics only measure the failure categories you've already named. A human reading traces finds the ones you haven't, like the agent that politely refuses a whole class of valid requests, or the one that retries a payment tool without checking the first attempt landed.\n\n## What should page you and what can wait\n\nPage a human for things that burn money or break trust quickly. That means a per-tool error rate above its baseline for ten minutes, a jump in cost per task, runs hitting the step cap, a sudden rise in zero-result retrievals, and any write action outside the scope you allowed. Quality signals such as judge scores, rephrase rate and thumbs-downs belong in a weekly review. They're sampled and noisy, so alert on rolling windows against the last release's baseline, or you'll train the team to ignore the channel.\n\nSome limits shouldn't be alerts at all. A runaway loop doesn't wait for someone to wake up. Set hard caps in code: a maximum number of steps per run, a maximum spend per run and per user per day, and a kill switch on destructive tools that requires explicit confirmation. Replit's answer after the SaaStr incident was to separate development and production databases and add a planning-only mode, which is the same idea. The agent shouldn't be the only thing standing between a bad decision and your production data.\n\n## A build order for a team of three\n\nYou don't need all of this on day one, and trying to build it all at once is how it never ships. In week one, get tracing on every run, with tool spans and the prompt version attached. In week two, add dashboards for tool errors, step counts, latency and cost per completed task, along with the code-level spend and step limits. In week three, write the deterministic checks and the two silent-failure ratios. In week four, switch on the judge for a sample and seed the golden set with the first real failures you've found.\n\nResist the urge to buy a platform's evaluation suite before the traces exist, because there's nothing for it to evaluate. Traces first, always. Everything else in this guide reads from them.\n\nOne practical detail to finish on: the OpenTelemetry generative AI conventions are still marked as in development, so attribute names can change between releases. Pin your instrumentation library versions and check the changelog before you upgrade, or a routine dependency bump can quietly empty your dashboards.\n\n**Also read:** [Your AI agent rollout payback period calculation is probably too short](https://startupfortune.com/your-ai-agent-rollout-payback-period-calculation-is-probably-too-short/) • [How to Prevent Prompt Injection in AI Agents Before Customers Do](https://startupfortune.com/how-to-prevent-prompt-injection-in-ai-agents-before-customers-do/) • [How to Calculate AI Agent ROI for Startups Before You Scale a Pilot](https://startupfortune.com/how-to-calculate-ai-agent-roi-for-startups-before-you-scale-a-pilot/)\n\n*This article is posted in [AI News](https://startupfortune.com/category/ai/), check it out for more related stories.*\n\n## Join the discussion\n\n[Open in the community →](https://startupfortune.com/community/)\n\nAlmost there. Sign in and your reply posts straight away.", "url": "https://wpnews.pro/news/how-to-monitor-ai-agents-in-production-before-a-bad-answer-reaches-a-customer", "canonical_source": "https://startupfortune.com/how-to-monitor-ai-agents-in-production-before-a-bad-answer-reaches-a-customer/", "published_at": "2026-11-13 07:54:45+00:00", "updated_at": "2026-11-13 08:19:57.452570+00:00", "lang": "en", "topics": ["ai-agents", "artificial-intelligence", "large-language-models", "mlops", "ai-safety"], "entities": ["Air Canada", "Replit", "Jason Lemkin", "SaaStr", "Amjad Masad", "Startup Fortune", "Fortune", "The Register"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/how-to-monitor-ai-agents-in-production-before-a-bad-answer-reaches-a-customer", "markdown": "https://wpnews.pro/news/how-to-monitor-ai-agents-in-production-before-a-bad-answer-reaches-a-customer.md", "text": "https://wpnews.pro/news/how-to-monitor-ai-agents-in-production-before-a-bad-answer-reaches-a-customer.txt", "jsonld": "https://wpnews.pro/news/how-to-monitor-ai-agents-in-production-before-a-bad-answer-reaches-a-customer.jsonld"}}