For now, I took a quick look at a few things:
This seems useful precisely because it does not present the 274 speech systems as though they had all been measured under one protocol. The distinction between the directly measured LLM leaderboard and the “surveyed, not benchmarked” TTS/STT research catalogs is clear, and the latter includes many of the things that tend to matter only after someone tries to use a system: deployment mode, licensing splits, lifecycle, pricing units, diarization, language coverage, and operational catches.
I am not a speech specialist, so I stayed with things I could check fairly narrowly against the public data and first-party documentation. I found a few concrete entries that may be worth refreshing, plus two small maintenance changes that might prevent the same class of issue from recurring.
The ASR page currently says:
surveyed, not benchmarked — 111 systems
in the heroListening surveyed · 106
in the pipeline/navigationasr.json
The JSON contains 106 system records, declares counts.total: 106
, and its category totals also sum to 106. TTS appears internally consistent at 168.
So this looks less like a substantive catalog problem and more like one static copy of the count surviving after the underlying data changed.
A small build-time invariant might be enough to prevent this:
systems.length
== counts.total
== sum(counts.by_category)
== hero count
== navigation / footer count
Alternatively, deriving every visible count from the same JSON value would remove the duplicated state entirely.
The current Fireworks row says that only the Whisper V3 Turbo rate could be confirmed on a Fireworks-owned page, while the commonly cited $0.0015/min
rate for Whisper V3 Large and the streaming prices could not.
The current Fireworks model library lists:
$0.0015 / Audio Minute
$0.0009 / Audio Minute
$0.0032 / Audio Minute
$0.0035 / Audio Minute
The older first-party audio transcription launch post also lists the $0.0015
and $0.0009
Whisper rates together, with some of the inference conditions around them.
I would not necessarily merge all four prices into one undifferentiated number: the two streaming systems appear to be separate products, and dedicated endpoints remain a different pricing path. But the narrower statement that the base Whisper price cannot be found on a Fireworks-owned page now seems refreshable.
The current row is understandably conservative: it records that Yandex’s documentation returned bot verification during the August 1 research pass, and therefore leaves pricing, languages, streaming, deployment options, and speaker labeling largely unverified.
The documentation is still somewhat awkward to retrieve automatically, but the current first-party pages expose enough information for a more complete row.
The SpeechKit STT overview describes an active service with:
The current supported languages and recognition models page explicitly lists 16 locale codes plus automatic language detection. It also documents three moving version tags:
general
general:rc
general:deprecated
and says the previous general
version is supported for two weeks after an update. That version-tag behavior may be particularly useful to record because it makes the hosted service a moving target even when the product name does not change.
There is also first-party documentation for speaker labeling. It is limited rather than general-purpose diarization:
FULL_DATA
modeSo I would not simply replace no-diarization
with an unrestricted diarization
label. Something like “limited two-speaker labeling under specific API v3 conditions” would be more precise.
Finally, SpeechKit Hybrid system requirements describe a customer-infrastructure deployment using Linux, Docker, and supported NVIDIA GPUs, including capacity figures for streaming recognition, file recognition, and file recognition with speaker labeling.
That creates a classification choice rather than one obviously correct edit:
If SpeechKit Hybrid is treated as the same product family:
update deployment / self-hosting notes in the existing row
If hosted SpeechKit and Hybrid are materially different purchasing products:
keep the hosted row and add Hybrid as a separate commercial on-prem entry
Either route seems more accurate than Cloud only
, but keeping hosted per-use pricing separate from individually licensed Hybrid deployment would avoid mixing unlike products.
The current Mistral Voxtral Transcribe 2 row already captures the important split reasonably well:
The row currently has no source_url
, but there are straightforward first-party sources.
The Voxtral Mini Transcribe 2 model card gives:
voxtral-mini-2602
$0.003/min
The Mistral audio overview distinguishes:
So this seems mostly like an opportunity to attach first-party links to the scopes already described, rather than a reason to rewrite the row.
Beyond those entry-level updates, two low-cost changes seem likely to return more value than a large schema redesign.
The 111/106 mismatch is exactly the kind of small error that becomes hard to notice in a generated static site because each individual artifact still looks reasonable.
A build check over the published artifacts could fail when:
JSON record count != declared JSON count
category total != declared total
HTML count != JSON count
README count != JSON count
The README check could perhaps remain advisory if it is intentionally hand-written, while JSON/HTML inconsistencies could fail the build.
The site and README link “Request a model” and “Report a correction” to GitHub’s new-issue page. At the moment, the repository’s Issues page says:
Issue creation is restricted in this repository
That may be completely intentional—for example, as a spam-control choice. GitHub only recently added an official collaborator-only issue creation setting, so I would not assume it is a configuration mistake.
However, the forum post already says corrections can also be collected in this thread. If the GitHub restriction is intentional, the smallest fix may simply be to make that existing fallback visible beside the correction button:
Report a correction:
- GitHub, if you have access
- otherwise use the Hugging Face Forum thread
That would preserve the repository setting without leaving ordinary readers at a dead end.
I would not say that provenance is generally missing. In a small static pass over the public data, the numerical ratings I found were already accompanied by a metric, named source, source URL, and as-of date. That is a good distinction to preserve.
The harder-to-maintain information appears to be the operational prose around those ratings:
For those fields, even a lightweight checked_at
and an optional claim-specific source may make later corrections much cheaper:
source_url: https://example.com/product
checked_at: 2026-08-04
claim_sources:
price:
url: https://example.com/pricing
checked_at: 2026-08-04
scope: serverless batch API
speaker_labeling:
url: https://example.com/docs/speaker-labeling
checked_at: 2026-08-04
scope: API v3, mono, maximum two speakers
This does not need to become a full provenance ontology. Applying it only to information expected to change would probably capture most of the benefit.
Hugging Face’s developing Evaluation Results format is one nearby example of keeping source, date, dataset/revision, notes, and contribution status distinct, although the catalog has a broader purpose than an evaluation-results file.
Overall, these look like normal maintenance pressure in a large, fast-moving catalog rather than a problem with the project’s basic approach.
My default order would be:
checked_at
only where the underlying fact is likely to change.That seems likely to improve the catalog’s maintainability without turning it into a much heavier system.