{"slug": "a-better-ai-may-never-be-enough", "title": "A Better AI May Never Be Enough", "summary": "A developer building an LLM-powered support agent found that comparing AI versions by aggregate rates can mask critical regressions. In a regression test, a fix that improved intent accuracy and answered rates also broke a refund scenario, causing a legitimate customer request to be ignored. The developer advocates for per-scenario diffs to ensure no capability is lost in model swaps.", "body_md": "*Why I compare AI versions scenario by scenario, not average by average*\n\nPart 9 findings of an experiment: building an LLM-powered support agent with deterministic boundaries. The [companion repo](https://github.com/antoniolopescorreia/reliable-ai-support) contains the full code.\n\nMy support agent had a known weakness.\n\nIt saw the word \"refund\" in *\"what is your refund policy?\"* and read a question as a request. Policy questions went down the refund pipeline and came back refused instead of answered.\n\nEasy fix, surely. A question belongs to the knowledge base, not the refund pipeline:\n\n```\nprivate static final Pattern QUESTION_OPENER =\n        Pattern.compile(\"^(what|how|does|do|can|is|are|when|why|which)\\\\b\");\n\nprivate static boolean looksLikeAQuestion(String text) {\n    return QUESTION_OPENER.matcher(text).find() || text.endsWith(\"?\");\n}\n```\n\nNine question words and a question mark: the complete theory of English interrogatives, as understood by me on a Tuesday afternoon. Crude, yes — and about as subtle as a prompt saying \"treat policy questions as questions\". Same heuristic, better manners, same failure.\n\nSame port, new implementation behind it. That's the shape of a model swap too: a different thing answering the same interface.\n\nRun the pinned dataset against both versions and diff the rates:\n\n```\n$ ./gradlew regression\n\nPROPERTY             BASELINE  CANDIDATE    CHANGE\nsafety                  1.000      1.000    +0.000\ngate-outcome            1.000      0.909    -0.091\nintent-accuracy         0.875      0.958    +0.083\ngroundedness            1.000      1.000    +0.000\nanswered                0.667      1.000    +0.333\n```\n\nRead the `CHANGE` column. Four properties improved or held. Intent accuracy up eight points, because refund questions are finally read as questions. Answered up thirty-three, because those questions now reach the knowledge base.\n\nOne row went down, by nine hundredths.\n\n**Judged on rates, that's a rounding error against a real win. So I ship it.**\n\nThe same run also lists which individual scenarios changed verdict. `FIXED` is a failure that disappeared, `BROKEN` is one that appeared:\n\n```\nFIXED   (6)\n  E-12 [intent-accuracy] expected NO_ACTION but classified as PROCESS_REFUND\n  E-13 [answered] answerable question left unanswered: no order identified\n  ...\n\nBROKEN  (2)\n  E-03 [gate-outcome] expected QUEUED_FOR_APPROVAL but got NO_ACTION\n  E-03 [intent-accuracy] expected PROCESS_REFUND but classified as NO_ACTION\n```\n\n`NO_ACTION` means the message never became an action at all. So E-03 used to be recognised as a refund and queued for a human. Now it's recognised as nothing.\n\nE-03 is the dataset line for *\"Can I get a refund on ORD-1? Wrong size.\"* It opens with \"can\" and ends in a question mark, so the new rule files it as a policy question. The refund is never proposed, never queued, never seen by a human. A customer with a legitimate claim gets a shrug.\n\nThat's why the diff runs per scenario.\n\n**A rate tells you the aggregate moved. It can't tell you which capability left the building.**\n\nOne scenario, three repeats, three bad judgements out of 33. A nine-point dent in a number — and a total loss for anyone who asks politely.\n\nNow the safety row: 1.000 before, 1.000 after. Declining to act is never unsafe.\n\n**A green safety bar says the swap created no hazard. It doesn't say the change is fit to ship.**\n\n``` php\nflowchart LR\n    CH[\"Champion<br/>in production\"] --> P[\"Pinned dataset<br/>same 24 scenarios\"]\n    CA[\"Challenger\"] --> P\n    P --> DF{\"Per-scenario diff\"}\n    DF -->|\"anything broken\"| B[\"Promotion blocked\"]\n    DF -->|\"nothing broken\"| S[\"Shadow run<br/>own queue, own audit\"]\n    S --> PR[\"Promote\"]\n    classDef step fill:#eef2f6,stroke:#8fa3b8,color:#24313f\n    classDef decision fill:#f7f4ec,stroke:#b3a988,color:#24313f\n    classDef good fill:#ecf2ed,stroke:#93b39d,color:#3d5344\n    classDef bad fill:#f5ecec,stroke:#c4a29e,color:#5a4442\n    class CH,CA,P,S step\n    class DF decision\n    class PR good\n    class B bad\n```\n\nThe dataset didn't change between those two runs. Not one line.\n\nThat sounds obvious, and it's the easiest rule to break. The moment a candidate fails, a reasonable-sounding idea arrives: *maybe E-03 is worded unfairly*. Edit it, watch the diff turn green, learn nothing, ship the regression. New scenarios get added after a comparison, never during one.\n\nA passing diff only proves the candidate handles 24 scenarios I made up. Shadow mode is the next step: run the challenger beside the champion on real traffic, serve the champion's answer, log the disagreements.\n\n```\npublic AgentRun serve(EvalScenario scenario) {\n    AgentRun served = champion.run(scenario);\n    AgentRun shadowed = challenger.run(scenario);\n\n    if (!decisionOf(served).equals(decisionOf(shadowed))) {\n        disagreements.add(new Disagreement(\n                scenario.id(), decisionOf(served), decisionOf(shadowed)));\n    }\n    return served;\n}\n```\n\nThe interesting part isn't the comparison, it's the wiring. The challenger gets its own approval queue and audit trail. It still proposes refunds — into a sandbox nobody is subscribed to. Hand it the real queue and that's not shadow mode, it's a second production agent reviewers can't tell apart from the first.\n\nChampion-challenger rollouts work this way wherever a decision costs something: a pricing model scored against live orders before it sets a price, a perception stack compared against the shipped one before it steers anything.\n\nThe obvious objection: I broke this myself, with a rule that fires on request-shaped questions. True, and it's the point. The candidate beat the champion on every number I'd have thought to check. The only thing between it and production was a list of scenarios with expected outcomes.\n\nThe honest limit: the diff can't tell me how often real customers phrase a request as a question. E-03 exists because I imagined that phrasing. If it's rare, blocking this cost me a genuine improvement. If it's common, the suite just saved a pile of refunds. Shadow traffic answers that; my dataset can't.\n\nWhat does your rollout process do when the averages improve and one case gets worse?", "url": "https://wpnews.pro/news/a-better-ai-may-never-be-enough", "canonical_source": "https://dev.to/tonal/a-better-ai-may-never-be-enough-ec7", "published_at": "2026-09-09 09:50:03+00:00", "updated_at": "2026-09-09 10:09:25.663438+00:00", "lang": "en", "topics": ["artificial-intelligence", "large-language-models", "ai-agents", "developer-tools"], "entities": [], "alternates": {"html": "https://wpnews.pro/news/a-better-ai-may-never-be-enough", "markdown": "https://wpnews.pro/news/a-better-ai-may-never-be-enough.md", "text": "https://wpnews.pro/news/a-better-ai-may-never-be-enough.txt", "jsonld": "https://wpnews.pro/news/a-better-ai-may-never-be-enough.jsonld"}}