Using LLMs for Customer Sentiment: Why Surveys Lie

A company using ChatGPT to analyze raw sales and support call transcripts discovered that customer sentiment from surveys is misleading, with transcripts revealing complaints about slow response times, overselling of features, and repeated explanations to multiple agents. The analysis, which prompted the LLM to strip away polite language, exposed operational failures that surveys had masked, leading to a monthly deep dive into unfiltered customer feedback.

Using LLMs for Customer Sentiment: Why Surveys Lie To find the disconnect, I dumped a massive batch of raw sales and support call transcripts into ChatGPT /en/tags/chatgpt/ . I didn't ask for a summary or a sentiment score; I asked it to tell me what customers actually think of us, specifically requesting that it strip away all the diplomacy. The results were brutal. While surveys called us "responsive," the actual call transcripts revealed customers complaining that we take days to reply. While surveys suggested the product was "great," the raw audio transcripts showed people feeling we oversell features and then walk them back later. The most damning part was the repetition—customers explicitly stating they had to explain the same problem to three different people before anything happened. This was a massive wake-up call in AI workflow implementation. A survey is a 10-second interaction where people are polite because they have a meeting to get to. A support call is where the unfiltered truth lives. By using an LLM agent to analyze raw text instead of sanitized checkboxes, we finally got a real-world view of our operational failures. I've shifted this into a monthly deep dive. It's impossible to ignore a systemic issue once you've read a customer describe it in their own words. For those wondering about the pipeline, the transcription tool is secondary to the prompt engineering. I used BuildBetter for the transcripts, though tools like Fathom or Otter do the same thing. The real value comes from how you prompt the LLM to ignore the "polite" layer of corporate communication and extract the actual pain points. Next OpenAI AI Keypad: Niche Tool or Productivity Win? → /en/threads/3682/