# Tests on 11 ASR models found several reproduced benchmark transcripts over audio > Source: > Published: 2026-08-22 08:12:36.335046+00:00 [Hugging Face](https://huggingface.co/blog/asr-benchmark-optimization) ### Tests on 11 ASR models found several reproduced benchmark transcripts over audio Which summary reads better? Pick one — models revealed after.Both summaries are AI-generated. Eleven leading open-source speech recognition models routinely output incorrect benchmark-specific transcripts even when the input audio directly contradicts them or has key words silenced. For production voice pipelines, this means top leaderboard scores severely overstate real-world transcription accuracy, leading to silent failures when deployed to actual users. To prevent shipping these fragile, over-optimized systems, you must bypass public ASR benchmarks and evaluate models using custom, held-out audio datasets. 11 open-source ASR models reproduced benchmark transcripts verbatim even when the audio contradicted them, exposing a 10–20% overstatement of real-world accuracy. This means your production pipelines that rely on leaderboard scores are silently shipping models that fail on basic phonetic fidelity—expect higher error rates in noisy, accented, or domain-shifted audio and plan for ensemble-based validation or held-out test sets before deployment.