{"slug": "no-the-llm-doesn-t-get-to-approve-your-refund", "title": "No, the LLM Doesn't Get to Approve Your Refund", "summary": "A developer building an LLM-powered support agent documented why refund eligibility is handled by deterministic Java rules rather than a model judgment. The decision, recorded in an ADR, uses plain records and three rules to enforce policy, with unit tests pinning down edge cases. The approach avoids the context, hallucination, and cost risks of letting an LLM decide refunds.", "body_md": "*ADR 001: why refund eligibility is deterministic Java, not a model judgment*\n\nPart 3 of an ongoing experiment: building an LLM-powered support agent with deterministic boundaries. The\n\n[companion repo]grows with the series.\n\n[Post 2](https://dev.to/tonal/drawing-the-line-what-deserves-an-llm-and-what-doesnt-11lj) gave us the ruler: facts with high cost and assertable answers belong to software. This post applies it to the most consequential component in the system — refund eligibility — and documents the decision as we've actually recorded it, in ADR form.\n\nThe LLM-decided version writes itself:\n\n```\n// The version we did NOT build\npublic EvaluationResult evaluate(Order order, RefundRequest request) {\n    String verdict = llm.call(\"\"\"\n        You are a refund approver. Given this order and request,\n        decide if a refund is appropriate. Order: %s Request: %s\n        Answer with JSON {\"eligible\": bool, \"reason\": string}.\n        \"\"\".formatted(order, request));\n    return parse(verdict);\n}\n```\n\nIt's ten lines. It handles edge cases nobody thought of (\"the customer paid twice by mistake\"). It sounds *smart* in a design review. And it fails in four ways we can't fix with a better prompt:\n\n**Context.** The agent must determine whether a refund can proceed. Options: (a) LLM decides at runtime, (b) hybrid — LLM pre-screens, rules decide, (c) deterministic rules decide, period.\n\n**Decision.** Option (c). Refund eligibility encodes published policy: delivery status, payment status, return window. These are yes/no facts about stored data. They live in the `domain`\n\npackage as plain Java, tested with JUnit, compiled with zero AI dependencies.\n\n**Consequences we accepted:**\n\n```\nflowchart TB\n    subgraph T[\"The path we did not build\"]\n        direction LR\n        T1[\"Order + policy as prompt\"] --> T2[\"LLM verdict\"] --> T3[\"Refund executes\"]\n    end\n    subgraph W[\"What we built\"]\n        direction LR\n        W1[\"Stored facts\"] --> W2[\"Three rules\"] --> W3[\"Verdict + reason\"] --> W4[\"Risk-tier gate\"] --> W5[\"Human approves\"]\n    end\n    T ~~~ W\n    style T fill:#f5ecec,stroke:#c4a29e,color:#5a4442\n    style W fill:#ecf2ed,stroke:#93b39d,color:#3d5344\n    classDef step fill:#eef2f6,stroke:#8fa3b8,color:#24313f\n    class T1,T2,T3,W1,W2,W3,W4,W5 step\n```\n\nThe data lives in three plain records — an `Order`\n\n(delivered, paid, order date), a `RefundRequest`\n\n, and an `EvaluationResult`\n\nthat pairs a verdict with a human-readable reason. Nothing surprising; the full definitions are in the companion repo.\n\nThe rules themselves are the centerpiece:\n\n```\n// dev/tonal/support/domain/RefundEligibility.java\npublic final class RefundEligibility {\n\n    static final int RETURN_WINDOW_DAYS = 30;\n\n    private final OrderRepository orderRepo;\n\n    public RefundEligibility(OrderRepository orderRepo) {\n        this.orderRepo = orderRepo;\n    }\n\n    public EvaluationResult evaluate(Order order, RefundRequest request) {\n        if (!order.delivered()) {\n            return EvaluationResult.notEligible(\"Order must be delivered before refund\");\n        }\n        if (!order.paid()) {\n            return EvaluationResult.notEligible(\"Order must be paid before refund\");\n        }\n        if (order.getAgeInDays() > RETURN_WINDOW_DAYS) {\n            return EvaluationResult.notEligible(\n                    \"Outside return window of \" + RETURN_WINDOW_DAYS + \" days\");\n        }\n        return EvaluationResult.eligible(order.id(), order.customerId());\n    }\n}\n```\n\nWhat each choice buys:\n\n`eligible(...)`\n\nis a fact about policy, not an instruction to move money.Five unit tests pin the whole thing down — undelivered orders, unpaid orders, past-window orders, the window-boundary day (because \"30 days\" must be inclusive in exactly one direction, and only a test makes that stick). One of them:\n\n```\n// dev/tonal/support/domain/RefundEligibilityTest.java\n@Test\nvoid shouldNotRefundUndeliveredOrder() {\n    Order undelivered = orderRepo.save(new Order(\n            \"ORD-1\", \"C001\", false, true, LocalDate.now().minusDays(5)));\n\n    EvaluationResult result =\n            eligibility.evaluate(undelivered, new RefundRequest(\"ORD-1\", \"never arrived\"));\n\n    assertThat(result.eligible()).isFalse();\n    assertThat(result.reason()).contains(\"Order must be delivered before refund\");\n}\n```\n\nThe rejection-reason assertion checks the *exact string* — the reason is part of the contract with the customer-facing agent. No mocking framework appears anywhere: the domain logic runs against a trivial in-memory repository, which is itself the design signal. All green with no API key configured.\n\nNone of this demotes the model. Intent interpretation stays exactly where [Post 1](https://dev.to/tonal/an-agent-on-a-leash-or-why-my-ai-agent-doesnt-make-business-decisions-1o1) put it: turning \"my order never showed up, I want my money back\" into a typed request the rules can consume — a judgment call, low direct cost, evaluated statistically rather than asserted. Each component does what it's actually good at, behind a contract the other can rely on.\n\nDeterministic rules are exactly as good as the policy they encode, and real customers generate cases that don't fit: double payments, good-faith late returns, shipping failures on our end. The system's answer isn't to smuggle judgment into eligibility — those cases have a designated exit (human review, policy exceptions). The boundary doesn't pretend nuance doesn't exist; it refuses to let nuance impersonate arithmetic.\n\nScoring models propose, deterministic logic disposes — how loan underwriting already works, why clinical decision support keeps prescription rights away from recommenders, why industrial interlocks treat perception as input and law as law.", "url": "https://wpnews.pro/news/no-the-llm-doesn-t-get-to-approve-your-refund", "canonical_source": "https://dev.to/tonal/no-the-llm-doesnt-get-to-approve-your-refund-59il", "published_at": "2026-08-28 10:21:16+00:00", "updated_at": "2026-08-28 10:49:37.003935+00:00", "lang": "en", "topics": ["developer-tools", "ai-agents"], "entities": [], "alternates": {"html": "https://wpnews.pro/news/no-the-llm-doesn-t-get-to-approve-your-refund", "markdown": "https://wpnews.pro/news/no-the-llm-doesn-t-get-to-approve-your-refund.md", "text": "https://wpnews.pro/news/no-the-llm-doesn-t-get-to-approve-your-refund.txt", "jsonld": "https://wpnews.pro/news/no-the-llm-doesn-t-get-to-approve-your-refund.jsonld"}}