A blueprint for reading every customer conversation.

BuildBetter, a customer-conversation analysis company founded by Spencer, has spent roughly $7M over six years building a system to read every customer conversation across an enterprise, according to a blueprint the founder shared. In BuildBetter's benchmark across 6,000 calls and 8,500 tickets, standard keyword and vector search retrieved only 11% of the relevant evidence on complex operational questions. The blueprint argues that dumping transcripts into ChatGPT or Claude, basic RAG search on a vector database, and generic AI classifiers all fail at scale, and recommends starting from the 20 questions a team argues about weekly.

A quick note before we start: This edition is a partnership with BuildBetter /go/buildbetter-deep-dive-top?p=web-note . I’ve known their founder, Spencer, for years and used multiple iterations of the product. What his team has built over the last five years is genuinely worth breaking down. I've known Spencer since around the time he shipped one of the first production products built on GPT-3, back in 2020. Over the six years since, his team has spent around $7M obsessing over a single problem: How do you read every single customer conversation across an entire company, and get fast, 100% reliable, useful answers from them? I asked Spencer to walk me through the architecture they built. He graciously agreed to let me share the blueprint both how to build the system yourself if you have the time and energy, and how to skip the headache entirely if you’d rather use what they built . If your company talks to customers all day... across sales calls, support tickets, and Slack... this is for you. If not, feel free to skip this email, no hard feelings. I'll have more for you tomorrow What you’re actually trying to build Imagine you run product or revenue, and on Monday morning you type: "Across enterprise accounts renewing this quarter, which high-severity issues came up in the last 30 days? Group identical issues together. For each, show me the accounts affected, their ARR, their health score, and whether anyone flagged the renewal as at risk. Link the exact moment they said it." Within seconds, you get back a clean, prioritized list, with the exact accounts, the revenue at stake, and a direct timestamp to the 30 seconds of audio where each customer voiced the complaint. Every B2B company already owns the raw material to answer this. It’s sitting untouched in Gong, Zendesk, Slack, and email. The problem is that nobody has the time to read thousands of hours of conversations. And the obvious ways people try to solve this with AI almost always fail. The three mistakes most teams make 1. Dumping transcripts into ChatGPT or Claude This works great for three call transcripts. It falls apart completely for three thousand. You quickly run into context limits, the model starts skimming, and your answer gets based on six random passages out of six thousand... delivered with total confidence. Plus, you’re paying the model to re-read your entire company history every time someone asks a question. 2. Slapping a basic search bot RAG on a vector database This takes a weekend to set up, and it handles simple lookups fine. If you ask "How many people asked for a HubSpot integration?" it works. But real questions are rarely that simple. Basic search looks for words that resemble your prompt. But customers don’t speak in clean labels. A customer on a call says: "I logged in yesterday and our dashboard had a big red billing banner." No part of that sentence says "bug" or "invoice failure." Standard search skips right past it, and it never shows up in your report. 3. Using generic AI classifiers Say you run a cybersecurity firm. If you ask a generic AI "what problems are customers reporting?" , every single transcript gets flagged. Why? Because your customers spend all day talking about "vulnerabilities" and "threats"... except they’re talking about your product's job, not a problem with your software. Without deep context on your specific business, AI can't tell the difference. In BuildBetter’s benchmark across 6,000 calls and 8,500 tickets, standard keyword and vector search retrieved only 11% of the relevant evidence on complex operational questions. What to do instead If you’re building this in-house, here is the playbook that actually works: 1. Start from the 20 questions your team argues about every week. Work backwards from the decisions that move the needle: churn risks, critical bugs, missing features, competitor pricing mentions. That list becomes your company's taxonomy. Every conversation must be mapped against these specific categories. 2. Process on intake, never on search. Don't scan raw transcripts when a user asks a question. Extract and categorize the signals the second a call finishes or a ticket closes. Your search queries should filter structured, pre-extracted data. That shifts the heavy compute cost to once-per-call, rather than paying to re-read everything on every single search. 3. Build your test set before you write the code. Take 100 to 200 real customer conversations and hand-label what the system should have caught. Whenever you tweak a prompt or change a pipeline step, run it against that benchmark. Otherwise, you’ll never know if a prompt tweak actually improved your accuracy or just broke something somewhere else. Do all three well, and you can get an internal system to around 70% accuracy on complex queries. For an internal prototype, that’s a great start. Why the last 29% cost BuildBetter $7M to nail A prototype working on your laptop is just step one. Then sales wants access, but security says they can't view support tickets. Then the CRM team renames a field and breaks your revenue reporting. Then your integrations break when Zoom or Zendesk ship an API update. And then someone has to maintain the infrastructure full-time. Then there's the accuracy gap. Getting from "mostly works" 70% to "executives make multi-million dollar roadmap bets on this data" 99% is where Spencer’s team spent the last five years and millions of dollars. They trained proprietary models specifically tuned to pull business signals out of messy customer conversations... so a "red billing banner" gets filed where it belongs, and a pen-test "vulnerability" isn't flagged as an outage. On the same benchmark where standard search found 11% of sources, their specialized engine hit 99%. To understand the types of use cases, let’s walk through what Brex, AppFolio, and PostHog , among others, do to run their sales, product, success, and retention playbooks on BuildBetter: - PostHog automatically pipes real customer quotes straight into GitHub issues so engineers see the exact context without having to sit through a call. - Brex automatically routes commitments made on sales calls to the right product managers so commitments made are commitments kept - Drip ran a single query and uncovered 91 separate requests for the same integration hidden inside six months of sales calls. Across their enterprise accounts, BuildBetter has influenced over $500M in pipeline and retained revenue. Build it or use it You have two paths: - If you want to build this internally: Spencer’s team has shared two resources to get you started: ProductOS /go/buildbetter-deep-dive-productos?p=web-resource , an open-source collection of AI skills for product work, and a product spec you can paste into your coding agent /go/buildbetter-deep-dive-spec?p=web-resource scroll to “Copy the spec” near the bottom . Use the skills to guide your work and the spec to try building your own version. - If you’d rather skip the infrastructure work: You can try BuildBetter directly. BuildBetter is a usage based product, so teams on average save 9x from alternatives and dramatically more from building in-house. They’re offering readers $1,000 in free credits , which covers roughly 200 hours of transcribed calls or about 8,000 support tickets... plenty to load a quarter’s worth of data and run the Monday morning test yourself. Try BuildBetter and get $1,000 in free credits → /go/buildbetter-deep-dive?p=web-cta Either way: pick the one hard question your leadership team has been guessing at. Ask it, inspect the raw evidence behind the answer, and see what you’ve been missing. Hit reply and let me know what you find. — Matt