{"slug": "best-of-n-tts-evaluation-is-confounded-by-asr-family-alignment", "title": "Best-of-$N$ TTS Evaluation is Confounded by ASR Family Alignment", "summary": "Researchers found that best-of-N text-to-speech evaluation is confounded by alignment between ASR verifier and evaluator families, with same-family pairs recovering 2-3× more oracle headroom than cross-family pairs. They propose cross-family rank ensembles that achieve 1.61% WER at N=10, a 12% relative improvement over F5-TTS, and recommend cross-evaluator triangulation as default reporting practice.", "body_md": "arXiv:2607.08256v1 Announce Type: new\nAbstract: Best-of-$N$ (BoN) inference improves content consistency in zero-shot text-to-speech by selecting from $N$ candidates with an automatic speech recognition (ASR) verifier. We identify an underexplored evaluation confound: a verifier's apparent quality depends strongly on which ASR family judges it. On LibriSpeech-PC test-clean~\\citep{librispeechpc} with F5-TTS~\\citep{f5tts}, verifier rankings reverse across Whisper, wav2vec~2.0, and HuBERT evaluators, and same-family verifier-evaluator pairs recover 2-3$\\times$ more oracle headroom than cross-family pairs despite near-identical representations (linear CKA $0.978$) -- a pattern consistent with identity- or lineage-level coupling rather than representational overlap. We propose two \\textbf{cross-family rank ensembles} (rank-averaging and conjunctive max-rank) that attain the lowest mean WER across three independent evaluators -- $1.61\\%$ at $N{=}10$ ($-12\\%$ relative to F5-TTS) -- with no measurable degradation under automatic SIM-o/UTMOS metrics; the best single verifier drives WER from $2.06\\%$ to $1.72\\%$ ($-16.5\\%$) under the official F5-TTS evaluator. We recommend cross-evaluator triangulation as default reporting practice.", "url": "https://wpnews.pro/news/best-of-n-tts-evaluation-is-confounded-by-asr-family-alignment", "canonical_source": "https://www.machinebrief.com/news/best-of-dollarndollar-tts-evaluation-is-confounded-by-asr-fa-rcln", "published_at": "2026-07-10 04:00:00+00:00", "updated_at": "2026-07-10 04:20:31.251507+00:00", "lang": "en", "topics": ["artificial-intelligence", "machine-learning", "natural-language-processing", "ai-research"], "entities": ["Whisper", "wav2vec 2.0", "HuBERT", "F5-TTS", "LibriSpeech-PC"], "alternates": {"html": "https://wpnews.pro/news/best-of-n-tts-evaluation-is-confounded-by-asr-family-alignment", "markdown": "https://wpnews.pro/news/best-of-n-tts-evaluation-is-confounded-by-asr-family-alignment.md", "text": "https://wpnews.pro/news/best-of-n-tts-evaluation-is-confounded-by-asr-family-alignment.txt", "jsonld": "https://wpnews.pro/news/best-of-n-tts-evaluation-is-confounded-by-asr-family-alignment.jsonld"}}