{"slug": "turning-closed-support-chats-into-a-claims-log-automatically", "title": "Turning Closed Support Chats Into a Claims Log, Automatically", "summary": "A team built a Claude Code bot that polls closed support chats, sends each transcript plus customer profile, order history and recent claim rows to Claude under a strict system prompt, validates the returned JSON, and writes a draft claims-sheet row at 3am for the agent to review. The bot processes three chats at a time, requests ten keys per claim (cause, detail, resolution, item, quantity, unit, quantity basis, delivery date, amount, note), and discards any result with more than eight items; it never edits a row a human has touched and logs every human cell change, which produced the pipeline's 20 rules. Three rules changed the numbers most: one row per product, treating \"not everything arrived\" as a missing item rather than a quality claim, and excluding zero-amount replacement-shipment orders so complaint counts don't double.", "body_md": "[Support automation](https://botsinproduction.com/blog#support)\n\n# Turning Closed Support Chats Into a Claims Log, Automatically\n\nA Claude Code bot reads every closed support chat, extracts the complaint and writes a claims-sheet row. The pipeline and the rules no tutorial mentions.\n\nBefore the bot, logging a customer complaint went like this: a support agent finishes the chat, opens the claims sheet, finds the customer, finds the product, types fifteen cells, and goes back to the next chat. Fifteen cells. Dozens of times a day. By people whose actual job is talking to customers, not data entry.\n\nRows were late. Rows were missing. The “cause” column said whatever the agent remembered at 6pm.\n\nNow a bot writes the first draft of every row, at 3am, while everyone is asleep and nobody can argue with it. This is how it works, what it gets wrong, and the rules that turned it from a cute demo into something the team actually trusts.\n\n## The pipeline\n\n1. **Poll for closed chats.** Our support tool has an API. We poll it instead of using webhooks. That’s a whole post, but the short version is: polling is restartable and webhooks are a promise the internet doesn’t always keep.\n2. **Build the context.** For each closed chat the bot pulls the transcript, the customer’s profile, their orders in the relevant window (and the previous 30 days if the window is empty), and the company’s most recent claim rows.\n3. **Summarise.** Transcript plus context goes to Claude with a strict system prompt. Three chats at a time.\n4. **Validate.** The JSON gets checked: a known cause category, product ids that exist on an order, at most eight items, amounts that actually parse as numbers.\n5. **Write.** Rows go into the sheet one at a time. A Slack message tags the agent who handled the chat with what got written.\n6. **Review.** A human fixes anything wrong. The bot never, ever edits a row a human has touched. It knows its place.\n\nAnd here’s the part I’m proudest of: the bot logs every cell a human changes afterwards. Those diffs are where every rule below came from.\n\n## The summariser prompt, in spirit\n\nThe system prompt asks for exactly these keys per claim: cause, detail, resolution, item, quantity, unit, quantity basis, delivery date, amount, note. Ten keys. If the model comes back with more than eight items, the result goes in the bin, because in practice that means it copied the order list instead of reading the complaint. (Models love a list. Give them a list and they will return the list.)\n\nThe two context additions that moved accuracy the most:\n\n- **The company’s recent claim rows.** The model learns what “normal” looks like for this customer, including how we’ve categorised their past claims.\n- **The order items.** When a customer says “the thing from Tuesday was bad”, the model needs to map that to a product id, and it can only do that if it can see what was ordered. We also compare phonetically, because transcribed phone calls spell product names however they sound that day.\n\n## The rules nobody tells you\n\nEvery rule came from a real row that was wrong and a colleague who noticed. There are twenty of them now, and they live in one place, [20 rules an AI support logger needs](https://botsinproduction.com/blog/20-rules-an-ai-support-logger-needs/), so I’ll only give the three that changed the numbers most:\n\n- **One row per product.** A chat about three bad products is three rows, not one row with a shopping list in it.\n- **“Not everything arrived” is a missing item, not a quality claim.** The model’s instinct is “quality” because the customer sounds unhappy. Unhappy is not a category.\n- **A zero-amount order is not a claim.** Replacement shipments appear as orders at unit price zero. Log them and your complaint count doubles overnight.\n\n## What goes to Slack\n\nEvery row produces a short message tagging the agent who handled the chat, plus me when the bot wasn’t sure. The chat link and the bot’s name go in their own columns, so the sheet’s “note” column stays for humans. Everything about the bot itself (an API error, a chat it gave up on) goes to my DM; the routing rules, including why a photo is shown only once, are in [why every bot alert goes to one DM](https://botsinproduction.com/blog/why-every-bot-alert-goes-to-one-dm/).\n\n## The night it fell over\n\nOn 2 October the bot gave up on seven chats after a few hundred `database is locked` errors. The cause was parallel sheet writes sharing one small SQLite file, and nothing about the model. The minute-by-minute is in [the night our bot gave up](https://botsinproduction.com/blog/database-is-locked-the-night-our-bot-gave-up/).\n\n## Is it accurate? The edit log says\n\nThe bot records every cell a human changes after it writes a row, so accuracy isn’t a feeling. Two snapshots, a month apart:\n\n|  | 6 Oct | 5 Nov | \n|---|---|---|\n| Rows the bot wrote | 276 | 1,080 | \n| Cells the bot filled | 2,945 | 12,812 | \n| Cells a human later changed | 335 (11.4%) | 1,628 (12.7%) | \n\nPer row that’s about twelve filled cells and, on average, one and a half corrected. On a typical weekday in late October the bot wrote 25 to 40 rows and about half of them got at least one cell changed, usually one or two cells, not the whole row.\n\nI expected the correction rate to fall as rules piled up. It hasn’t, and I’ve stopped treating that as the target. The target is that every correction either becomes a rule or turns out to be a judgment call the bot shouldn’t make. What did change is the work: the team now *corrects* a cell or two instead of *typing* fifteen, and the rows are waiting when they arrive with their coffee.\n\n## One line from me\n\nThe team now corrects rows instead of writing them. That shift, not the summary, is what the bot is for.\n\nThe phone half of the inbox, where the customer talks instead of types, is in [summarising support calls](https://botsinproduction.com/blog/summarising-support-calls-with-speech-to-text-and-claude/).", "url": "https://wpnews.pro/news/turning-closed-support-chats-into-a-claims-log-automatically", "canonical_source": "https://botsinproduction.com/blog/turning-support-chats-into-a-claims-log-automatically/", "published_at": "2026-10-21 00:00:00+00:00", "updated_at": "2026-11-06 01:21:27.493615+00:00", "lang": "en", "topics": ["ai-agents", "ai-tools", "large-language-models", "artificial-intelligence"], "entities": ["Claude", "Claude Code", "Slack"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/turning-closed-support-chats-into-a-claims-log-automatically", "markdown": "https://wpnews.pro/news/turning-closed-support-chats-into-a-claims-log-automatically.md", "text": "https://wpnews.pro/news/turning-closed-support-chats-into-a-claims-log-automatically.txt", "jsonld": "https://wpnews.pro/news/turning-closed-support-chats-into-a-claims-log-automatically.jsonld"}}