No, the LLM Doesn't Get to Approve Your Refund

A developer building an LLM-powered support agent documented why refund eligibility is handled by deterministic Java rules rather than a model judgment. The decision, recorded in an ADR, uses plain records and three rules to enforce policy, with unit tests pinning down edge cases. The approach avoids the context, hallucination, and cost risks of letting an LLM decide refunds.

ADR 001: why refund eligibility is deterministic Java, not a model judgment Part 3 of an ongoing experiment: building an LLM-powered support agent with deterministic boundaries. The companion repo grows with the series. Post 2 https://dev.to/tonal/drawing-the-line-what-deserves-an-llm-and-what-doesnt-11lj gave us the ruler: facts with high cost and assertable answers belong to software. This post applies it to the most consequential component in the system — refund eligibility — and documents the decision as we've actually recorded it, in ADR form. The LLM-decided version writes itself: // The version we did NOT build public EvaluationResult evaluate Order order, RefundRequest request { String verdict = llm.call """ You are a refund approver. Given this order and request, decide if a refund is appropriate. Order: %s Request: %s Answer with JSON {"eligible": bool, "reason": string}. """.formatted order, request ; return parse verdict ; } It's ten lines. It handles edge cases nobody thought of "the customer paid twice by mistake" . It sounds smart in a design review. And it fails in four ways we can't fix with a better prompt: Context. The agent must determine whether a refund can proceed. Options: a LLM decides at runtime, b hybrid — LLM pre-screens, rules decide, c deterministic rules decide, period. Decision. Option c . Refund eligibility encodes published policy: delivery status, payment status, return window. These are yes/no facts about stored data. They live in the domain package as plain Java, tested with JUnit, compiled with zero AI dependencies. Consequences we accepted: flowchart TB subgraph T "The path we did not build" direction LR T1 "Order + policy as prompt" -- T2 "LLM verdict" -- T3 "Refund executes" end subgraph W "What we built" direction LR W1 "Stored facts" -- W2 "Three rules" -- W3 "Verdict + reason" -- W4 "Risk-tier gate" -- W5 "Human approves" end T ~~~ W style T fill: f5ecec,stroke: c4a29e,color: 5a4442 style W fill: ecf2ed,stroke: 93b39d,color: 3d5344 classDef step fill: eef2f6,stroke: 8fa3b8,color: 24313f class T1,T2,T3,W1,W2,W3,W4,W5 step The data lives in three plain records — an Order delivered, paid, order date , a RefundRequest , and an EvaluationResult that pairs a verdict with a human-readable reason. Nothing surprising; the full definitions are in the companion repo. The rules themselves are the centerpiece: // dev/tonal/support/domain/RefundEligibility.java public final class RefundEligibility { static final int RETURN WINDOW DAYS = 30; private final OrderRepository orderRepo; public RefundEligibility OrderRepository orderRepo { this.orderRepo = orderRepo; } public EvaluationResult evaluate Order order, RefundRequest request { if order.delivered { return EvaluationResult.notEligible "Order must be delivered before refund" ; } if order.paid { return EvaluationResult.notEligible "Order must be paid before refund" ; } if order.getAgeInDays RETURN WINDOW DAYS { return EvaluationResult.notEligible "Outside return window of " + RETURN WINDOW DAYS + " days" ; } return EvaluationResult.eligible order.id , order.customerId ; } } What each choice buys: eligible ... is a fact about policy, not an instruction to move money.Five unit tests pin the whole thing down — undelivered orders, unpaid orders, past-window orders, the window-boundary day because "30 days" must be inclusive in exactly one direction, and only a test makes that stick . One of them: // dev/tonal/support/domain/RefundEligibilityTest.java @Test void shouldNotRefundUndeliveredOrder { Order undelivered = orderRepo.save new Order "ORD-1", "C001", false, true, LocalDate.now .minusDays 5 ; EvaluationResult result = eligibility.evaluate undelivered, new RefundRequest "ORD-1", "never arrived" ; assertThat result.eligible .isFalse ; assertThat result.reason .contains "Order must be delivered before refund" ; } The rejection-reason assertion checks the exact string — the reason is part of the contract with the customer-facing agent. No mocking framework appears anywhere: the domain logic runs against a trivial in-memory repository, which is itself the design signal. All green with no API key configured. None of this demotes the model. Intent interpretation stays exactly where Post 1 https://dev.to/tonal/an-agent-on-a-leash-or-why-my-ai-agent-doesnt-make-business-decisions-1o1 put it: turning "my order never showed up, I want my money back" into a typed request the rules can consume — a judgment call, low direct cost, evaluated statistically rather than asserted. Each component does what it's actually good at, behind a contract the other can rely on. Deterministic rules are exactly as good as the policy they encode, and real customers generate cases that don't fit: double payments, good-faith late returns, shipping failures on our end. The system's answer isn't to smuggle judgment into eligibility — those cases have a designated exit human review, policy exceptions . The boundary doesn't pretend nuance doesn't exist; it refuses to let nuance impersonate arithmetic. Scoring models propose, deterministic logic disposes — how loan underwriting already works, why clinical decision support keeps prescription rights away from recommenders, why industrial interlocks treat perception as input and law as law.