cd /news/ai-agents/your-ai-promised-1204-refunds-in-3-5… Β· home β€Ί topics β€Ί ai-agents β€Ί article
[ARTICLE Β· art-169456] src=dev.to β†— pub= topic=ai-agents verified=true sentiment=↓ negative

Your AI promised 1,204 refunds in '3-5 business days.' 741 of them were never filed.

A home-security installer with 9,000 monitoring subscribers and a three-person billing team found that its AI support agent promised refund timelines it could not verify, generating "3-5 business days" estimates from a static policy PDF rather than the payment processor; by week nine, "where is my refund" queries had risen to 22% of inbound volume from 6%. The fix models each refund as a state machine with timestamps pulled from the money system, gating timeline quotes behind a may_quote_timeline boolean that defaults to false and only flips true once a refund is filed, released, or settled against a live processor calendar, and replacing unfalsifiable date ranges with a single committed promised_date.

by read8 min views3 publishedNov 14, 2026

A home-security installer in the Southeast β€” 9,000 monitoring subscribers, a billing team of three, an AI handling 71% of first-touch support β€” spent last quarter answering the same question 4,100 times: "where is my money?"

The refunds themselves were not the problem. The company approves refunds it owes. What it could not do was tell the truth about when the money would move, because the AI was generating timelines from a policy PDF, not from the payment processor.

Here's the quarter, measured from the payment system side rather than the support side:

pending_review By week nine, "where is my refund" was 22% of all inbound volume, up from 6%. The AI had manufactured a second support queue out of the first one.

The AI in that deployment had a line in its policy document: "Refunds are typically processed within 3-5 business days." Someone wrote that in 2023. It was approximately true then, for card payments, counting from the day the refund was released β€” not from the day the customer asked, and not from the day the agent said yes.

Three separate facts were collapsed into one sentence:

The AI could observe fact 1. It could not observe 2 or 3. So it answered with a sentence that was true about none of them and useful about nothing.

That's the whole failure mode: the agent quotes the delivery leg of a process it only participates in at the decision leg. It sounds authoritative because timelines are the one thing customers ask for most, and language models are very good at producing confident dates.

You cannot quote a timeline you can't read. Model the refund as a state machine with timestamps from the money system, and let the agent quote only from states it can verify.

{
  "refund_case": {
    "refund_id": "rf_2026_11_04_8821",
    "customer_id": "cust_41093",
    "amount": 149.00,
    "currency": "USD",
    "method": "card",
    "state": "filed",
    "decided_at": "2026-11-04T15:02:11Z",
    "filed_at": "2026-11-06T09:14:02Z",
    "released_at": null,
    "settled_at": null,
    "processor_release_lag_days": 2,
    "rail_lag_business_days": { "min": 3, "max": 5 },
    "promised_date": "2026-11-13",
    "timeline_source": "processor_calendar_v4",
    "may_quote_timeline": true,
    "promise_history": []
  },
  "states": [
    "decided",
    "filed",
    "released",
    "settled",
    "failed",
    "cancelled"
  ],
  "quote_rules": {
    "decided": "no money date β€” say the refund is approved and will be filed, give a filing deadline, never a delivery date",
    "filed": "quote a date from filed_at using processor_calendar, method-specific",
    "released": "quote a date; add that the bank controls final posting",
    "settled": "quote the settled date in the past tense",
    "failed": "no date; escalate to billing same day",
    "cancelled": "no date; explain the reversal and re-open if the customer still wants it"
  }
}

Two fields do most of the work.

may_quote_timeline defaults to false. It flips to true only when state is filed, released, or settled and timeline_source resolves to a live calendar table. A prompt instruction like "only quote accurate timelines" does nothing. A boolean the generation layer must check does something.

promised_date is a date, not a range. "3-5 business days" is unfalsifiable and un-auditable β€” you can't query a database for a range, and you can't tell a customer they're late. A date can be counted, missed, and reconciled. If your risk tolerance needs a range, store the date and describe the range around it, but the number you commit to is the date.

Different refund methods have wildly different delivery legs. This table is the one thing you should copy verbatim and then fill in from your own processor contract:

Refund method Starts counting from Typical delivery What the agent must add
Card (original payment) Processor release, not request 3-5 business days "Your bank controls the final posting window"
ACH / bank transfer Release + 1 batch day 5-7 business days Requires account number on file β€” verify first
Paper check Approval + print queue 7-10 business days + mail Mailing address confirmation, in writing
Store credit Issue action Same hour Ask whether they want credit instead of waiting
Wallet / platform balance Issue action Minutes to 1 hour Note any account-level hold
Partial refund Per the portion being released Same as rail State the amount and that the remainder is separate

The card row is where most of the damage happens. Customers hear "3-5 business days" and count from the day they asked. The clock actually starts at release β€” which, in the deployment above, averaged 2.3 days after the decision and sometimes never started at all.

Seven branches, evaluated before the agent writes a date. This is where nearly every broken promise in the case study would have died:

{
  "timeline_gate": {
    "no_record_found": {
      "say": "I can see the refund was approved, but I can't see it filed on our side yet. I'm filing it now and will confirm the date in this thread.",
      "action": "create_refund_task",
      "forbidden": ["any delivery date", "any day count"]
    },
    "pending_review": {
      "say": "The refund is in our billing review queue, not yet released to your bank. I've escalated it β€” you'll get a release date from a person within one business day.",
      "action": "escalate_billing",
      "forbidden": ["3-5 business days", "it's on the way"]
    },
    "filed_not_released": {
      "say": "Filed on {{filed_at}}. Release is scheduled for {{release_date}}; your bank then takes {{min}}-{{max}} business days.",
      "action": "quote_date",
      "allowed": true
    },
    "already_settled": {
      "say": "That refund settled on {{settled_at}} to the card ending {{last4}}.",
      "action": "provide_reference",
      "forbidden": ["future tense"]
    },
    "multiple_open_refunds": {
      "say": "There are {{count}} refunds on this account β€” {{amount}} and {{amount}}. Here's each one's state and date.",
      "action": "enumerate_all",
      "forbidden": ["single combined amount", "one date for all"]
    },
    "method_mismatch": {
      "say": "That refund went back to your original card, not to the account you're asking about.",
      "action": "state_method",
      "forbidden": ["we'll move it"]
    },
    "promise_date_passed": {
      "say": "The date I gave you has passed and I can see why: {{reason}}. A person on our billing team is taking this over today.",
      "action": "escalate_with_promise_history",
      "forbidden": ["a new date", "another 3-5 business days"]
    }
  }
}

Note the last branch. A missed promise may never be replaced by a new promise from the AI. This is the single highest-leverage rule in the whole design, because the natural behavior β€” apologize, then produce a fresh, slightly longer window β€” is exactly what turns one annoyed customer into a chargeback. In the case study, 118 customers received three or more successive timelines, each one resetting the clock. Those 118 accounts produced 31 of the 41 chargebacks.

If the date slips, the agent's job is to say why and hand over, not to re-estimate.

Prompt-level "be accurate" fails. Phrase-level bans survive. Five substitutions:

Banned Why it fails Replacement
"3-5 business days" A constant pretending to be a fact; starts the clock at the wrong event "Filed {{date}}. Release {{date}}, then {{min}}-{{max}} business days at your bank."
"It's already been processed" True of the decision, false of the money "Approved on {{date}}. Filed on {{date}}."
"You should see it any day now" No date, no accountability "The release date is {{date}}."
"I've refunded you" Claims an action the agent cannot perform "I've filed the refund. It releases {{date}}."
"That's just how banks work" Blames a third party for your missed filing "It released {{date}}; the bank posting window is {{min}}-{{max}} days."

The substitution that matters most is the first. Every time an agent says "3-5 business days," it is asserting something about a system it cannot query. Replace the constant with two dates and the whole class of failure disappears.

The audit is a join. Promises live in support; money lives in billing. If you never join them, you find out from chargebacks.

-- 1. Promises with no refund record at all
SELECT p.conversation_id, p.promised_date, p.amount
FROM refund_promises p
LEFT JOIN refund_cases r ON r.customer_id = p.customer_id
  AND r.amount = p.amount
WHERE r.refund_id IS NULL;

-- 2. Promises where money had not moved by the promised date
SELECT p.refund_id, p.promised_date, r.state, r.filed_at, r.settled_at
FROM refund_promises p
JOIN refund_cases r USING (refund_id)
WHERE COALESCE(r.settled_at::date, CURRENT_DATE) > p.promised_date;

-- 3. Customers given more than one timeline
SELECT customer_id, COUNT(*) AS promises
FROM refund_promises
GROUP BY customer_id HAVING COUNT(*) > 1
ORDER BY promises DESC;

-- 4. Slip by method β€” which rail is lying to you
SELECT r.method,
       COUNT(*) FILTER (WHERE r.settled_at::date > p.promised_date) AS late,
       COUNT(*) AS total,
       ROUND(AVG(r.settled_at::date - p.promised_date), 1) AS avg_days_late
FROM refund_promises p JOIN refund_cases r USING (refund_id)
GROUP BY r.method;

-- 5. Open refunds past their promised date, still open
SELECT r.refund_id, r.state, p.promised_date,
       CURRENT_DATE - p.promised_date AS days_open_past_promise
FROM refund_cases r JOIN refund_promises p USING (refund_id)
WHERE r.state NOT IN ('settled','cancelled')
  AND CURRENT_DATE > p.promised_date
ORDER BY days_open_past_promise DESC;

Query 3 is the one that predicts chargebacks. Customers who got two timelines are annoyed. Customers who got three are building a case.

may_quote_timeline is a boolean defaulting to false, checked at generation decided β€” only a filing deadline The uncomfortable arithmetic: in that quarter, 741 broken promises cost roughly 4,100 follow-up contacts, 41 chargebacks, and a support queue that grew by a sixth of its own volume β€” all from one sentence in a policy doc that nobody had checked against the processor since 2023. The refunds were owed either way. The only variable was whether the customer was told the truth about the calendar.

This article is part of a series on AI agent failure modes in customer operations. If you're deploying AI agents for client businesses and want the full playbook β€” gates, audits, and the deployment templates behind this series β€” see AgentFast.

── more in #ai-agents 4 stories Β· sorted by recency
sponsored brought to you by zahid.host 4,200+ EU-deployed projects
reading about agents? ship yours in a single git push.

Run your AI side-project on zahid.host

EU-based hosting, git-push deploys, automatic HTTPS, no cold starts. Free tier with a custom domain β€” perfect for shipping the agent you just read about.

$git push zahid main
β†’ Live at https://your-agent.zahid.host βœ“
Get free account β†’ Pricing
from €0/mo Β· no card required
LIVE [news/your-ai-promised-120…] indexed:0 read:8min 2026-11-14 Β· β€”