Deepgram endpointing=300 Cut Off 22% of My Voice AI Users Mid-Answer

A developer found that setting Deepgram's endpointing parameter to 300ms caused 22% of long user answers to be cut off mid-thought in their voice interview practice tool, Preterview. Analysis of 1,140 long answers showed a median mid-answer pause of 0.9s and a p90 of 2.4s, with 97% of answers containing at least one pause longer than 300ms. A two-gate fix reduced cutoffs to 3.1% while preserving most of the latency gains.

The user said, "The biggest trade-off I made on that project was..." and then paused to think. The bot jumped in with "Great, thanks for sharing Next question." That pause was about 700 milliseconds. My Deepgram endpointing was set to 300. So as far as my pipeline knew, the person was done talking. I had spent two weeks making my voice agent respond faster. It worked. Turns went out quicker than ever. Then I pulled a week of logs and found that 22% of long answers had been cut off mid-thought . I had built a very fast machine for interrupting people. This is the autopsy, the numbers, and the two-gate fix that got cutoffs down to 3.1% without giving back most of the latency. endpointing parameter sets how many milliseconds of silence end a turn. Low values like 300ms make voice agents feel fast but cut people off whenever they pause to think. endpointing=300 . Thinking pauses had a median of 0.9s and a p90 of 2.4s, so 300ms never stood a chance. Deepgram endpointing is a silence timer on the streaming speech-to-text side. When the voice activity detector hears no speech for endpointing milliseconds, Deepgram finalizes the transcript and marks it with speech final: true . Most voice agent loops, including mine, treat speech final as "the user is done, your turn." Here's roughly what my connection looked like: wss://api.deepgram.com/v1/listen ?model=nova-2 &interim results=true &endpointing=300 &utterance end ms=1500 &vad events=true And the naive turn logic: python async def on transcript msg : if msg "is final" : buffer.append msg "channel" "alternatives" 0 "transcript" if msg.get "speech final" : user turn = " ".join buffer .strip buffer.clear await agent.respond user turn bot starts talking The problem is the assumption baked into that if . speech final means "silence happened." It does not mean "the thought is finished." For quick commands "play the next song" those are the same thing. For anyone explaining something, they aren't. Because people pause to think far longer than 300ms, especially when the question is hard. Once I measured my own traffic, the gap was obvious. I pulled every user turn from one week that ran longer than 10 seconds of speech. That gave me 1,140 "long answers." Then I looked at the silence gaps inside each answer, meaning pauses followed by more speech from the same person within a few seconds. | Metric | Value | |---|---| | Long answers analyzed | 1,140 | | Median mid-answer pause | 0.9s | | p90 mid-answer pause | 2.4s | | Answers with at least one pause 300ms | 97% | | Answers cut off by the bot | 251 22% | That 97% row is the one that hurt. Almost every long answer had a pause that my config considered a finished turn. The only reason the cutoff rate wasn't higher is that interim transcripts and network jitter sometimes bought people a few extra hundred milliseconds by accident. Count "barge-backs": cases where the user starts speaking again within 2.5 seconds after the bot's turn has begun. Nobody cuts off a bot two seconds in unless the bot cut them off first. It's a proxy, so I validated it. I hand-labeled 200 sessions by listening to the recordings. The barge-back signal matched my labels in 181 of them. The misses were mostly people saying "sorry, go ahead" or coughing, which the detector counted as speech. Good enough to trust the trend, not good enough to trust the second decimal place. This is where the story gets specific. The system I'm describing is Preterview, a voice interview practice tool that runs mock interviews with different interviewer styles and writes up a report afterward full disclosure: I built it, Preterview https://preterview.com/en . Interview answers are close to the worst case for endpointing: long, structured, and full of "let me think about how to put this" pauses. Cutting someone off mid-answer in a mock interview isn't a small UX glitch, it wrecks the thing they came to practice. Raising endpointing to 1200ms dropped cutoffs from 22% to 6%. It also added about 900ms of dead air to every turn, including the ones where the user was obviously done. I ran it for two days. The cutoff graph looked great. The session recordings sounded like talking to someone on a satellite phone. Short answers like "Yes, twice" were followed by more than a second of nothing, and a few users started saying "hello?" into the silence, which then got transcribed as a new turn. I had traded one kind of awkward for another. The real lesson: a single silence threshold is the wrong knob. Silence length alone can't tell "done" from "thinking." utterance end ms alone? Deepgram also sends an UtteranceEnd event based on word timings, controlled by utterance end ms . I tried waiting for it instead of speech final , with it set to 1500. It behaved about like endpointing=1200 : fewer cutoffs, slower everywhere. It is better at ignoring background noise, which helped for people in noisy rooms. But it has the same blind spot. It measures time, not meaning. A two-gate turn detector. Keep endpointing=300 as a fast tentative end of turn, then decide whether the transcript looks finished before committing. Meanwhile, start drafting the reply so a real end costs almost nothing. Gate 1: a cheap heuristic 0ms . If the transcript ends with a conjunction or filler "and", "so", "because", "um", "like" , or is suspiciously short for the question type, don't commit. Wait for UtteranceEnd . Gate 2: a small LLM classifier ~150-250ms . If the heuristic passes, a small fast model gets the question and the transcript and answers one token: COMPLETE or INCOMPLETE . python async def on speech final turn text, question : if ends with continuation turn text : return await wait for utterance end draft = asyncio.create task agent.draft reply turn text speculative verdict = await classify completeness question, turn text if verdict == "COMPLETE" and not user resumed speaking : await agent.speak await draft else: draft.cancel thrown away, costs tokens await wait for utterance end The speculative draft is what keeps latency sane. By the time the classifier says "complete," the reply is often already half-generated. Same week-long measurement, same barge-back detector, same answer length filter: | Config | Cutoff rate | Median added delay | |---|---|---| | endpointing=300 baseline | 22% | 0ms | | endpointing=1200 | 6% | ~900ms | | utterance end ms=1500 | ~7% | ~950ms | | Two-gate + speculative draft | 3.1% | +140ms | The honest parts: Look at your pause distribution before you pick a number. Ten minutes with a histogram of mid-turn silences would have saved me a week. If your users give short commands, 300ms is great. If they explain things, your p90 pause is probably measured in seconds, and no single threshold will be both fast and polite. And measure interruptions directly. Latency dashboards reward cutting people off, because a bot that interrupts has, technically, the lowest response time in the building. For conversational voice AI where users give long answers, yes: in my logs, Deepgram endpointing at 300ms cut off 22% of long answers because real thinking pauses had a median of 0.9s and a p90 of 2.4s. Raising the threshold to 1200ms fixes cutoffs but adds about 900ms to every turn. The better approach is to treat the 300ms speech final as a tentative end, check completeness with a trailing-word heuristic and a small LLM classifier, and draft the reply speculatively in parallel. That brought my cutoff rate to 3.1% for a median cost of 140ms. Written by the developer behind Preterview https://preterview.com/en , an interview prep platform.