{"slug": "open-asr-leaderboard-adds-egyptian-and-nigerian-english-what-changes-for-speech", "title": "Open ASR Leaderboard Adds Egyptian and Nigerian English: What Changes for Speech-to-Text Tests", "summary": "Hugging Face announced on November 6, 2026 a collaboration with Liva AI that adds Egyptian English (1.34 hours) and Nigerian English (1.55 hours) as private, held-out conversational test sets on the Open ASR Leaderboard, alongside a new accent breakdown in the Private tab comparing US versus non-US and Inner Circle versus Outer and Expanding Circles. The leaderboard's headline average word error rate does not change, consistent with Hugging Face's May 2026 decision to keep the default average on public data only. The addition matters for teams choosing speech-to-text models for voice agents, call transcription or captions with global users, because it tests accents models were not tuned for.", "body_md": "The Open ASR Leaderboard now tests two more English accents with data models cannot have trained on. On November 6, 2026, Hugging Face announced a collaboration with Liva AI that adds **Egyptian English (1.34 hours)** and **Nigerian English (1.55 hours)** as *private*, held-out conversational test sets. The Private tab also gets a new accent breakdown: US versus non-US, and Inner Circle versus Outer and Expanding Circles. The leaderboard's headline average does not change.\n\nIf you pick a speech-to-text model for a product with global users, this is a useful change to read. It is small in hours of audio, but it tests the thing that public benchmarks tend to hide: how a model handles accents it was not tuned for. The [announcement is on Hugging Face](https://huggingface.co/blog/open-asr-leaderboard-liva?ref=explainx).\n\n## TL;DR: what changed and what did not\n\n| Question | Answer | \n|---|---|\n| What was added? | Two private test sets: Liva AI Conversational EG and NG. | \n| How much audio? | 1.34 hours (Egyptian) and 1.55 hours (Nigerian). | \n| What style? | Conversational, spontaneous; punctuated, cased, with disfluencies. | \n| Who made it? | Liva AI, with Hugging Face (Eric Bezzam, Ashley Mo, Aoi Otani). | \n| New view? | Accent breakdown in the Private tab (US vs non-US; Kachru's Three Circles). | \n| Does the default average change? | Not announced. In May, Hugging Face kept the default average on public data. | \n| Who should care? | Teams choosing ASR for voice agents, call transcription or captions with global users. | \n\n## What is the Open ASR Leaderboard?\n\nIt is Hugging Face's public ranking of speech recognition models by word error rate (WER) and speed. According to Hugging Face's May 2026 post, it launched in September 2023 and has been visited over 710,000 times. Two ideas hold it together, in the team's words: **standardization** (all test sets gathered on the Hub, and a normalizer based on Whisper's that removes punctuation and casing and maps to American spelling) and **openness** (UI code and evaluation scripts are open source).\n\nBoth help and hurt. Standard, open tests make results comparable. They also make it easy to tune a model to the test, a habit the team calls \"benchmaxxing.\" A model can score well and still fail on real audio. That tension is why private data keeps coming up.\n\n## Why does private test data matter?\n\nIn May 2026, the team published [\"Adding Benchmaxxer Repellant to the Open ASR Leaderboard\"](https://huggingface.co/blog/open-asr-leaderboard-private-data?ref=explainx), opening with Goodhart's Law: \"When a measure becomes a target, it ceases to be a good measure.\" Appen and DataoceanAI supplied 11 private splits covering scripted and conversational speech in five accents, and Hugging Face kept them hidden to reduce the risk of test-set contamination.\n\nTwo design choices from that post matter for how you read the new Egyptian and Nigerian sets:\n\n- **The default average stays public-only.** The post said Hugging Face was not updating the average WER; users could toggle private datasets in to see the effect.\n- **Vendors that sell training data are a risk, one commenter argued.** In the comments, a reader (weiwchu) raised the concern that the two data vendors also offer datasets to ASR providers for training. Hugging Face's Eric Bezzam replied that the private data is excluded from the default average WER.\n\nLiva AI is a third provider. Hugging Face describes it as a company that collects multimodal interaction data through consumer products used by real people, with natural, high-context behavior rather than artificially elicited tasks. That is the company's own description. Adding a new provider also fits the team's stated mitigation: use several providers so no one source gives a lasting edge.\n\n## What is new in the November update?\n\nOpen ASR Leaderboard private data accent breakdown by English accent\n*Image: [Hugging Face](https://huggingface.co/blog/open-asr-leaderboard-liva?ref=explainx)*\n\nThe two new sets are both conversational, with transcripts that keep punctuation, casing and disfluencies (the \"um\" and false starts of real speech). That makes them harder and closer to call audio than a scripted read-aloud.\n\nThe **accent breakdown** is the other half of the update. Using Kachru's Three Circles of English, the leaderboard will now report:\n\n1. **US versus non-US** , where non-US means UK, Australia, Canada, India, Nigeria and Egypt.\n2. **Inner Circle versus Outer and Expanding Circles.** The Inner Circle is the US, UK, Australia and Canada. The Outer and Expanding Circles are India, Nigeria and Egypt.\n\nHugging Face credits Hume AI's leaderboard for the idea, and says it will split the Outer and Expanding Circles further as more accents arrive. Hugging Face says Indian English joined earlier through a private split made with Voice Arena.\n\n## How should I read the numbers when they appear?\n\nTreat the new sets as a stress test, not a verdict.\n\n- **Small samples.** 1.34 and 1.55 hours of audio is a few speakers at most. A model that looks great or poor here may owe it to one voice, one microphone or one topic. The post does not say how many speakers each set has.\n- **Conversational only.** There is no scripted counterpart yet for these accents, so you cannot separate \"accent\" from \"spontaneous speech.\" A drop could come from either.\n- **One metric.** WER hides a lot. Names, numbers and code-switching can matter more to your product than the average.\n- **No scores in the post.** The announcement does not name top models or publish scores. Go to the leaderboard Space for results, and remember the post gives no information on who leads.\n\nA practical test: pick the three models you are considering, run them on 20 minutes of your own audio from your actual users, and compare. Use the leaderboard to shortlist, not to decide.\n\n## Why does this matter for people who build with voice?\n\nVoice products are shipping fast. We have covered new speech-to-text launches from [OpenAI's GPT Live Transcribe](https://explainx.ai/blog/openai-gpt-live-transcribe-gpt-transcribe-july-2026), [Google's Gemini 3.5 Transcribe](https://explainx.ai/blog/gemini-3-5-transcribe-speech-to-text-launch-august-2026), [Grok Voice Transcribe 2](https://explainx.ai/blog/grok-voice-transcribe-2-launch-2026) and [Meta Muse Voice transcribe](https://explainx.ai/blog/meta-muse-voice-transcribe-realtime-speech-to-text-2026). Each comes with vendor-chosen numbers. A third-party leaderboard with held-out data in under-represented accents is one of the few checks that the vendor does not control.\n\nTwo cautions for builders:\n\n- **Accent coverage is still narrow.** Two more accents is progress, not coverage. Hugging Face itself calls it \"continued effort.\" If your users speak Hindi-English, Arabic dialects or Nigerian Pidgin, test directly.\n- **Private does not mean unbeatable.** If a model vendor buys data from the same providers, the edge can come back over time. This is the concern a commenter raised in May; the default average excludes private data.\n\nFor a wider view of why leaderboards drift and what to do about it, see our explainers on [Goodhart's Law and benchmark contamination](https://explainx.ai/blog/goodharts-law-ai-benchmark-contamination-gsm1k-mmlu-arena-2026) and [Google DeepMind's double-blind evaluation work](https://explainx.ai/blog/google-deepmind-double-blind-ai-evaluation-benchmark-contamination-august-2026). If you work with on-device dictation, our look at [Superwhisper's S1 Mini transcript normalizer](https://explainx.ai/blog/superwhisper-s1-mini-on-device-transcript-normalizer-august-2026) shows how much the text-normalization step alone can change WER-style results.\n\n## What is word error rate, and why do accents move it?\n\nWord error rate counts the words a model gets wrong (substitutions, deletions and insertions) divided by the number of words in the reference transcript. A WER of 10 percent means roughly one word in ten is off. Because the leaderboard normalizes both the model output and the reference (no punctuation, no casing, American spelling), it compares words, not formatting.\n\nAccents move WER for simple reasons. A model learns the sounds, rhythms and vocabulary it saw in training. If most of its training audio is American English, other accents land further from what it knows, and errors rise. Spontaneous speech adds fillers, restarts and overlapping talk, which scripted audio lacks. The new sets combine both pressures: non-US accents and unscripted conversation.\n\nThe May post makes a point worth repeating here. In the team's words from their report, there is no single catch-all ASR model: some do better on American English, others on diverse accents and multilingual settings, others are tuned for speed or conversational audio. A model that scores lower on one axis is not necessarily a worse model overall. So the right question is not \"which model is first\" but \"which model is best on the audio my users produce.\"\n\n## How can I contribute or follow along?\n\n- **Use the leaderboard:** open the Open ASR Leaderboard Space on Hugging Face and switch to the Private tab to see the accent breakdown.\n- **Suggest a dataset or model:** the open_asr_leaderboard GitHub repository is the place to propose additions; check its contribution notes first.\n- **Read the method:** the May post explains the normalizer, the scripted versus conversational splits and why the average is public-only.\n\n## What to do this week\n\n1. If you run ASR in production with African or Middle Eastern English speakers, add the Private tab to your model-selection checklist.\n2. Build a 20-minute internal test set from consented, real user audio, and score candidates on it.\n3. Log WER by accent group, not only overall. A good average can hide a bad tail.\n4. Re-check in a few weeks. Hugging Face says more accents will join and the Outer and Expanding Circle group will be split.\n\n## Related reading\n\n*Accurate as of November 6, 2026. The announcement gives no model scores; check the leaderboard Space for current results.*", "url": "https://wpnews.pro/news/open-asr-leaderboard-adds-egyptian-and-nigerian-english-what-changes-for-speech", "canonical_source": "https://explainx.ai/blog/open-asr-leaderboard-egyptian-nigerian-english-liva-private-data-2026?utm_source=rss&utm_medium=feed&utm_campaign=blog", "published_at": "2026-11-06 00:00:00+00:00", "updated_at": "2026-11-06 11:23:28.841208+00:00", "lang": "en", "topics": ["artificial-intelligence", "machine-learning", "natural-language-processing"], "entities": ["Hugging Face", "Liva AI", "Open ASR Leaderboard", "Eric Bezzam", "Ashley Mo", "Aoi Otani", "Appen", "DataoceanAI"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/open-asr-leaderboard-adds-egyptian-and-nigerian-english-what-changes-for-speech", "markdown": "https://wpnews.pro/news/open-asr-leaderboard-adds-egyptian-and-nigerian-english-what-changes-for-speech.md", "text": "https://wpnews.pro/news/open-asr-leaderboard-adds-egyptian-and-nigerian-english-what-changes-for-speech.txt", "jsonld": "https://wpnews.pro/news/open-asr-leaderboard-adds-egyptian-and-nigerian-english-what-changes-for-speech.jsonld"}}