{"slug": "defects-missed-in-transcription-ai-speaks-after-0-5-second-silence", "title": "Defects Missed in Transcription — AI Speaks After 0.5-Second Silence", "summary": "A developer at forge.workstyle.tech discovered that speech-to-text (STT) based quality control for text-to-speech (TTS) models can miss defects where the model produces sounds not in the script after a 0.5-second silence. The developer found that Whisper STT dropped these short utterances, leading to a false 100% pass rate, and developed a waveform-based detection method using RMS envelope segmentation to catch such artifacts.", "body_md": "📝 Originally published (in Japanese) at\n\n[forge.workstyle.tech].\n\nI used to perform quality control (QC) for TTS models using this process:\n\nI created 12 voices and passed all of them through this QC. Whisper got 4/4 accuracy, no trailing elongation, and sound pressure was within normal range. I reported **100% pass rate**.\n\nLater, when I rechecked from a different angle, **4 of them still had defects**. These were invisible to STT-based inspection due to fundamental limitations.\n\nThe first clue came when I received this report:\n\n```\nご覧ください。   Total: 1.61s  Body: 0.88s → Silence: 0.48s → 【0.16s utterance】\nこちらです。     Total: 1.65s  Body: 0.72s → Silence: 0.56s → 【0.28s utterance】\n```\n\nAfter finishing the script, there’s a full 0.5-second silence followed by a 0.1–0.3 second utterance. This isn’t trailing resonance—**the model is producing sounds not in the script** (the root cause was training corpus contamination: [\"3 characters\" allowed by the quality gate became verbal tics](https://forge.workstyle.tech/blog/three-chars-became-a-verbal-tic/)).\n\nThe reason my initial inspection missed this is simple: **Whisper dropped these sounds**.\n\n```\nご覧ください。   → STT: \"ご覧くださいああ\"     ← barely caught\nこちらです。     → STT: \"こちらです\"           ← completely dropped\n```\n\nA 0.28-second utterance doesn’t appear in the transcription at all. Short sounds that aren’t meaningful words may not appear in STT output. **As long as you’re only looking at transcriptions, this defect doesn’t exist.**\n\nI even concluded, “STT got 0/6, so no extra sounds,” mistaking the blind spot of my measurement method for a property of the target.\n\nWhat’s actually being output appears in the waveform. By extracting voiced blocks from the RMS envelope and examining their sequence, we can detect these artifacts.\n\n``` python\ndef segments(wav_bytes, thr_ratio=0.06):\n    \"\"\"Returns [(start_sec, end_sec), ...] of voiced blocks\"\"\"\n    w = wave.open(io.BytesIO(wav_bytes)); sr = w.getframerate()\n    x = np.frombuffer(w.readframes(w.getnframes()), dtype=np.int16) / 32768\n\n    W, H = int(sr * 0.020), int(sr * 0.010)          # 20ms window / 10ms hop\n    rms = np.array([np.sqrt(np.mean(x[i*H:i*H+W]**2))\n                    for i in range(max(0, (len(x)-W)//H))])\n\n    # Use the larger of relative or absolute threshold\n    act = rms > max(rms.max() * thr_ratio, 0.004)\n\n    segs, s = [], None\n    for i, a in enumerate(act):\n        if a and s is None:\n            s = i\n        elif not a and s is not None:\n            if (i - s) * 0.010 >= 0.03:              # Ignore blocks <30ms\n                segs.append((s * 0.010, i * 0.010))\n            s = None\n    if s is not None:\n        segs.append((s * 0.010, len(act) * 0.010))\n    return segs\n```\n\nThe `max(rms.max() * 0.06, 0.004)`\n\npart is subtly critical.\n\n**Relative threshold alone** fails for low-volume voices. If the overall volume is quiet, the maximum value is small, causing noise floor to be misclassified as voiced.\n\n**Absolute threshold alone** fails for high-volume voices. Breathing or lip smacks get classified as voiced.\n\nThe 12 voices had sound pressure ranging from −13.3 to −18.8 dB, so neither threshold alone could work across all voices.\n\n**Discarding blocks under 30ms** is also necessary. Without this, lip noise or quantization noise appears as many tiny blocks, breaking downstream logic.\n\nOnce voiced blocks are extracted, we check: “Is there sufficient silence before the final block, and does that block have sufficient duration?”\n\n```\nGAP_MIN  = 0.25      # Silence this long or more indicates a separate utterance\nTAIL_MIN = 0.06      # Duration this long or more indicates an artifact\n\ndef has_trailing_artifact(wav):\n    segs = segments(wav)\n    if len(segs) < 2:\n        return None                      # No artifact if only one block\n    gap  = segs[-1][0] - segs[-2][1]     # Silence before last block\n    tail = segs[-1][1] - segs[-1][0]     # Duration of last block\n    if gap >= GAP_MIN and tail >= TAIL_MIN:\n        return (gap, tail)\n    return None\n```\n\n`GAP_MIN=0.25`\n\nseparates natural trailing resonance or pauses from clearly separated utterances. Measured artifacts had silence gaps of 0.26–0.91 seconds, so 0.25 is sufficient.\n\n`TAIL_MIN=0.06`\n\navoids catching fade-out tails. Measured artifacts were 0.07–0.36 seconds long.\n\nIn my first scan, **all 12 models triggered the detector**. The probe sentence contained this:\n\n```\nでは、始めます。\n  Block[0] = \"では\"\n  Block[1] = \"始めます\"   ← 0.40s silence followed by 0.75s duration\n```\n\nThis was a pause after a comma. After “では、” there’s a gap, then “始めます。” follows. The final block is part of the script itself, yet it perfectly matches the detection condition (gap + subsequent utterance).\n\n**The condition “there’s utterance after the final gap” will always produce false positives for sentences containing commas.** That’s because it doesn’t consider script structure.\n\nThere are two fixes:\n\n**Limit probes to single sentences.** If you exclude sentences with commas, any utterance after the body can be definitively identified as an artifact. This is what I adopted—simple implementation and no dependency on the script.\n\n**Align with script end position.** Derive the script end position from Whisper segments and check if energy exists beyond that point. This is more general but reintroduces STT dependency. If artifacts don’t appear in Whisper segments, the end position might be incorrectly determined.\n\nAfter removing false positives:\n\n```\nBefore (with commas): 29 / 72 detections   ← all 12 models triggered\nAfter (single sentences only): 16 / 72 detections   ← only 4 models triggered\n```\n\n| Model | Artifacts |\n|---|---|\n| Male Narrator | 6/6 |\n| Female Operator | 4/6 |\n| Female Presenter | 4/6 |\n| Male Presenter | 2/6 |\n| Remaining 8 | 0/6 |\n\nHad I reported the initial results as-is, I would have spread **false panic of “all 12 failed.”** Once you build a detector, you **must first test it on things that should not trigger it**.\n\nWhat I learned is that **audio quality inspection requires multiple methods that reveal different layers**:\n\n| Method | Reveals | Misses |\n|---|---|---|\n| STT (transcription) | Word omissions, substitutions, large insertions | Short artifacts, silence structure, audio quality |\n| Waveform envelope | Utterance boundaries, silence, artifacts | What it’s saying |\n| Acoustic features (F0, sound pressure, intonation) | Pitch, volume, variation | Correctness of content |\n| Listening test | Everything (but subjective & not scalable) | — |\n\nRelying only on STT for QC meant **assuming everything visible in the most familiar tool would be visible everywhere**. In reality, STT only shows “what can be recognized as words.”\n\nInterestingly, these 4-second artifacts are **hard to notice even when listening**. A 0.1-second sound feels like “some lingering resonance” unless you’re paying close attention. Only by laying out the numbers do you realize there’s an abnormal structure: “silence 0.5s followed by sound.”\n\n**If humans can’t perceive it subjectively, machines must measure it. And every measurement method has its own blind spots.**\n\nReflecting on this experience, here’s the process I should have followed:\n\nThe fourth point is the key lesson: **when detection rates are too high, suspect the detector, not the targets**.\n\nA record of designing voices from single captions, manufacturing training corpora, and mass-producing role-specific practical voices. This article is **Part 3: Quality Gate**.\n\n← Previous: [Weeding out candidates using fixable defects](https://forge.workstyle.tech/blog/measuring-factory-defects-as-product-traits/)\n\n→ Next: How 70 minutes of training material vanished in an instant due to a network blink\n\nFull series (18 parts)\n\nThe insights in this article are compiled in the [Diffusion TTS Manufacturing Pipeline for Mass-producing Practical Voices](https://forge.workstyle.tech/blog/diffusion-tts-manufacturing-pipeline/).", "url": "https://wpnews.pro/news/defects-missed-in-transcription-ai-speaks-after-0-5-second-silence", "canonical_source": "https://dev.to/orca_forge/defects-missed-in-transcription-ai-speaks-after-05-second-silence-3id5", "published_at": "2026-09-04 00:14:59+00:00", "updated_at": "2026-09-04 00:23:42.701777+00:00", "lang": "en", "topics": ["machine-learning", "ai-tools", "developer-tools"], "entities": ["Whisper", "forge.workstyle.tech"], "alternates": {"html": "https://wpnews.pro/news/defects-missed-in-transcription-ai-speaks-after-0-5-second-silence", "markdown": "https://wpnews.pro/news/defects-missed-in-transcription-ai-speaks-after-0-5-second-silence.md", "text": "https://wpnews.pro/news/defects-missed-in-transcription-ai-speaks-after-0-5-second-silence.txt", "jsonld": "https://wpnews.pro/news/defects-missed-in-transcription-ai-speaks-after-0-5-second-silence.jsonld"}}