📝 Originally published (in Japanese) at
[forge.workstyle.tech].
When I had the trained voice model read "こんにちは" (Hello), it stretched the phrase to "こんにちわぁ." There was no instruction to stretch it in the script.
The feedback was as follows:
For "こんにちは," it's pronounced as "こんにちわぁ" with an accent on the last syllable. It feels like something is mixed in.
"Something mixed in" was accurate, and indeed, something was mixed in. The training corpus contained clips with stretched endings.
The problem was that the mechanism to detect it was fundamentally non-functional by design.
In corpus generation, the audio read by TTS is transcribed using Whisper and then compared with the script.
def _kana(s: str) -> str:
return "".join(chr(ord(c) - 0x60) if "ァ" <= c <= "ヶ" else c for c in s)
_PUNCT_RE = re.compile(r"[、。！？!?…・\s「」ー〜,\.]")
_REPEAT_RE = re.compile(r"(.)\1+")
def _collapse(s):
return _REPEAT_RE.sub(r"\1", _PUNCT_RE.sub("", s or ""))
def judge_transcript(script_text, transcript, ...):
a = _kana(_collapse(script_text))
b = _kana(_collapse(transcript))
sm = difflib.SequenceMatcher(None, a, b)
...
Normalization is performed before comparison. It’s a straightforward implementation.
Here, look at _PUNCT_RE
. Among the characters to be removed is ** ー (prolongation mark)**. And
_REPEAT_RE
compresses consecutive identical characters into one.
Script: こんにちは
Transcription: こんにちわー
After normalization:
Script → こんにちは
Transcription → こんにちわ ← The `ー` is removed
The match rate is high. It becomes a difference of just one character between "は" and "わ." The information that the ending was stretched is discarded during normalization.
The same happens with consecutive vowels.
Transcription: こんにちわあ → _REPEAT_RE compresses consecutive "あ" → こんにちわ
This means this verification cannot detect stretched endings no matter what. The normalization written to ignore prolongation marks works the same way even when we want to detect them.
The normalization itself is correct. If the goal is to absorb variations in notation and check for content consistency, prolongation marks should be removed. The problem was that there was only one normalization for two purposes.
I separated the judgment for content consistency from the judgment for stretched endings. The latter uses the raw string before normalization.
_TAIL_LONG_RE = re.compile(r"[ーぁ-ん]$")
def trailing_elongation_mismatch(script_text: str, raw_transcript: str) -> bool:
"""Detects stretched endings not present in the script.
⚠️ Pass the raw transcription from Whisper to `raw_transcript`.
Kana normalization discards prolongation marks, so detection is impossible with normalized strings.
"""
script = (script_text or "").rstrip("。、！？!? ")
trans = (raw_transcript or "").rstrip("。、！？!? ")
if not script or not trans:
return False
tail_script = script[-1]
if "ー" not in script and trans.endswith("ー"):
return True
if len(trans) > len(script) and trans[len(script)-1:].startswith(tail_script):
extra = trans[len(script):]
if extra and all(c in "ぁぃぅぇぉあいうえおー" for c in extra):
return True
return False
With the judgments separated, the caller checks them independently.
res = judge_transcript(text, tr["text"]) # Content consistency (with normalization)
tail = trailing_elongation_mismatch(text, tr["text"]) # Stretched endings (raw string)
if tail:
continue # If the ending is stretched, immediately redraw (even if the content matches)
if res.ok:
save(wav)
Stretched endings are disqualified even if the content matches. No matter how high the content consistency, if the ending is stretched, it’s not included in the training material. Relaxing this would ingrain the habit.
However, setting it to zero completely reduces yield. In emotionally charged speech, some stretching naturally occurs.
Ultimately, I used this two-condition OR logic:
def accept(text, transcript):
v = judge_transcript(text, transcript)
tail = trailing_elongation_mismatch(text, transcript)
return (v.ratio >= 0.82 and tail <= 2) or (v.ratio >= 0.70 and tail == 0)
This ensures that clips with "questionable content and stretched endings" are reliably discarded. It tolerates recognition degradation due to emotional expression while not allowing the stretching habit to pass.
Another effective measure was writing it in the caption during generation.
When designing role-specific voices, I included this in the caption:
A female announcer's voice accurately reading a news script. Clear and easy to understand,
with a calm, intellectual tone, pronouncing each word distinctly, including the endings.
The last part, "pronouncing each word distinctly, including the endings," was key.
The effect was clear. Measuring 24 candidates × 5 probe sentences = 120 clips showed zero stretched endings. Before the quality gate could reject them, stretched audio simply wasn’t generated.
Role-included caption (distinct endings) → Stretched endings 0/120
The gate is a mechanism to "discard bad ones," but discarding reduces yield. If it can be prevented upstream, that’s cheaper. For TTS that can be instructed via captions, it’s worth writing items the quality gate checks into the caption as well.
As I pursued this phenomenon, a deeper structure emerged. The desirability of stretched endings varies by use case.
So, I defined "speech styles" for each use case and varied the corpus script and quality gate strictness by speech style. For business-style speech, the ending gate is strictly applied; for casual styles, it’s relaxed.
What’s important is that speech style is baked into the corpus and cannot be changed during synthesis (as discussed in [[speaking-style-is-baked-into-the-corpus|speaking speed cannot be changed after training]]). If you want both "tight endings" and "stretched endings" for the same voice, bake two versions with the same design values (caption + seed) but different speech styles. That’s what I’m doing.
Separate normalization by purpose. Normalization for content consistency and normalization for detecting specific anomalies are different. Trying to do both with one normalization causes one to fail.
Be aware of discarded information. Including ー
in _PUNCT_RE
was the right decision, but a judgment needing that information arose later. Adding comments to normalization about "what is being discarded" helps the next person notice.
It’s cheaper to prevent it upstream. Discarding at the gate reduces yield. If it can be instructed during generation, hit it there.
Symptoms appear in model behavior. Even if the corpus data shows "a few clips with stretched endings," it doesn’t look abnormal. The habit only appears after training and speaking. Dataset inspection alone is insufficient; a process to confirm post-training output is necessary.
This is a record of designing voices from a single caption, manufacturing training corpora, and mass-producing role-specific practical voices. This article is Part 3: Quality Gate.
← Previous: [[one-rough-clip-ruins-the-whole-style|One Rough Clip Ruins the Whole Style]]
→ Next: [[the-character-that-broke-the-tts-input|"少々" Becomes "しょも" — Permitted Character List Was Trimming Japanese]]
All 18 Articles in the Series
The insights are summarized in [[Manufacturing Pipeline for Mass-Producing Practical Voices from Diffusion TTS]].