Whisper Hallucinations: 'Thanks for Watching' in 9% of My Transcripts

A user emailed me a screenshot of their feedback report. Under "Communication," my scoring model had written: "Closed the answer with 'Thanks for watching,' which feels scripted and out of place in a live conversation."

They never said it. They d for about four seconds, decided they were done, and stopped talking. Whisper filled the silence for them, and then an LLM graded them on the words Whisper made up.

This post covers Whisper hallucinations on silence: how often they happened in my pipeline (9.2% of clips), why the built-in no_speech_threshold didn't catch them, and the config change that got the count from 183 to 4.

no_speech_threshold alone won't save you.no_speech_prob is high avg_logprob is low. Confident hallucinations pass.vad_filter=True (Silero VAD) with speech_pad_ms=400, condition_on_previous_text=False, plus a small guard on known phrases. 183 dropped to 4. I run Preterview, a voice interview practice app (full disclosure: I built it). You talk to an AI interviewer, and afterward you get a written report scored from your transcript. Live turn-taking runs on a streaming ASR, but the report uses a second, slower pass: every answer clip goes through faster-whisper with large-v3 on a single GPU, and the cleaned transcript goes to Claude for scoring.

That second pass is where accuracy matters, so I thought it was the safe part. It was the part lying to me.

In my data, about 1 clip in 11. I pulled 2,000 answer clips from one week of sessions and flagged anything matching a list of known hallucination phrases, plus any segment with compression_ratio > 2.4 (a decent signal for repetition loops). That flagged 196 clips. I listened to all 196. It took an afternoon and a lot of coffee.

13 were real: people genuinely saying "thank you" at the end. The other 183 were invented.

The location pattern was the giveaway. 164 of the 183 sat in the first or last segment of the clip, right next to at least two seconds of silence. When I split by trailing silence:

Interview answers are basically designed to trigger this. People think before they speak, and they trail off when they finish. My recorder kept capturing a few seconds after the last word so it wouldn't clip anyone. That safety margin was the hallucination fuel.

Whisper's decoder is a language model, and it was trained on a huge amount of weakly labeled web audio paired with captions. When the audio gives it almost nothing, the decoder falls back on its prior: what text usually appears over quiet, music, or a fade-out in its training data? Video outros. "Thanks for watching." Caption credits.

It also always works on 30-second windows. A short clip gets padded, so the model is constantly looking at stretches of nothing and being asked to produce text for them. Most of the time it emits nothing or a no-speech token. Sometimes it confidently writes a sign-off.

The repetition loops have a separate cause. With condition_on_previous_text=True (the default), the previous segment's text gets fed in as a prompt. Once one segment goes wrong, the next one is conditioned on the wrong text, and you get "I think that I think that I think that" until the window ends.

This one cost me the most time. I assumed no_speech_threshold=0.6 meant "drop anything Whisper thinks is 60%+ likely to be silence." It doesn't. Here's the logic from openai-whisper's transcribe.py (faster-whisper does the same thing):

if no_speech_threshold is not None:
    should_skip = result.no_speech_prob > no_speech_threshold
    if logprob_threshold is not None and result.avg_logprob > logprob_threshold:
        should_skip = False

A segment only gets skipped if no-speech probability is high and the average log-probability of the decoded text is low. If the model is confident about its made-up text, the segment survives.

And "Thanks for watching" is something Whisper is very confident about. Across my 183 hallucinations, the median no_speech_prob was 0.71, comfortably above the 0.6 threshold. But 129 of them had avg_logprob above -1.0, so the skip got overruled.

What I tried before the real fix:

logprob_threshold=None Thresholds were the wrong layer. The problem was that silence reached the decoder at all.

Stop sending silence to the model. faster-whisper ships a built-in Silero VAD filter that strips non-speech regions before decoding. Here's the config I landed on:

from faster_whisper import WhisperModel

model = WhisperModel("large-v3", device="cuda", compute_type="float16")

KNOWN_GHOSTS = {
    "thanks for watching", "thank you for watching", "thank you",
    "please subscribe", "subtitles by the amara.org community",
}

def transcribe_answer(path: str) -> str:
    segments, _ = model.transcribe(
        path,
        beam_size=5,
        vad_filter=True,
        vad_parameters=dict(min_silence_duration_ms=500, speech_pad_ms=400),
        condition_on_previous_text=False,
    )
    kept = []
    for seg in segments:
        norm = seg.text.strip().lower().rstrip(".!")
        if norm in KNOWN_GHOSTS and seg.no_speech_prob > 0.5:
            continue  # confident ghost in a quiet region
        kept.append(seg.text.strip())
    return " ".join(kept)

I added changes one at a time and re-ran the same 2,000 clips after each:

Turning off conditioning killed the loops (27 down to 3). VAD did the heavy lifting on everything else.

The VAD had its own failure mode: with default padding, 11 clips lost their first word. Soft openers like "So," or "Um, yeah" got classified as noise. Bumping speech_pad_ms to 400 brought that to 3 and let 2 hallucinations back in. I took that trade.

The phrase guard is deliberately narrow. It only drops a known phrase when the segment's own no_speech_prob is above 0.5. Of the 13 real "thank you"s, 12 had no_speech_prob under 0.2 and survived. One person said it so quietly that it got dropped. I can live with that.

Two were "Thank you." in clips with loud keyboard noise, where VAD called the typing speech. One was a lone "you." One was a three-time "um" loop that survived because conditioning wasn't the cause. I stopped there. Going from 4 to 0 would mean more aggressive VAD, and that eats real words.

Two things, both cheap.

The scorer prompt now knows transcripts are machine-made. I added a line telling Claude the transcript came from automatic speech recognition, may contain artifacts like sign-offs or repeated phrases, and must never penalize those. Even with perfect ASR config, a judge that trusts its input blindly is one bad clip away from writing nonsense about a real person.

Hallucination rate is now a metric. A nightly job runs the phrase list and the compression-ratio check over the day's transcripts and alerts if the flagged rate goes above 1%. It sits around 0.3%. If a model upgrade or a recorder change pushes it back up, I'll know before a user emails me a screenshot.

A bonus I didn't expect: VAD made transcription faster. Skipping silent regions cut average GPU time per clip from 0.84s to 0.61s. The fix paid for itself.

Whisper says "Thanks for watching" on silence because its decoder is a language model trained on web video captions, and when the audio carries no speech it falls back to the text that most often appears over quiet endings in that data: video outros and caption credits. The default no_speech_threshold doesn't stop it, since a segment is only skipped when no-speech probability is high and decoding confidence is low, and these phrases decode with high confidence. In my pipeline that meant 183 of 2,000 clips (9.2%) contained invented text. The reliable fix is to remove silence before decoding: enable vad_filter=True in faster-whisper with some speech padding, set condition_on_previous_text=False to stop repetition loops, and add a narrow guard for known phrases in high no-speech segments. That took my count from 183 to 4.

Written by the developer behind Preterview, an interview prep platform.