{"slug": "anyspeechrecognizer-and-anydecisionmodel-open-speech-and-decision-models-for", "title": "AnySpeechRecognizer and AnyDecisionModel: Open Speech and Decision Models for Swift Apps", "summary": "Hugging Face engineer Mattt announced on October 29, 2026 two Swift packages, AnySpeechRecognizer and AnyDecisionModel, that give Apple-platform developers one familiar Swift API with a choice of open model behind it, pulling models from the Hugging Face Hub and running them on-device. AnySpeechRecognizer mirrors Apple's SpeechAnalyzer, SpeechTranscriber, DictationTranscriber, SpeechDetector and AssetInventory so code can switch engines by changing the import from `Speech` to `AnySpeechRecognizer`, and adds a SpeakerDiarizer running NVIDIA's Streaming Sortformer on Core ML that tracks up to four speakers, which Apple's Speech framework does not offer. The packages require Swift 6.2+, macOS 26 or iOS 26+, and Apple silicon for local runtimes, and the authors' benchmarks come from a single Mac with no iPhone testing.", "body_md": "Hugging Face engineer Mattt has [announced two Swift packages](https://huggingface.co/blog/anydecisionmodel?ref=explainx) for Apple platforms: AnySpeechRecognizer and AnyDecisionModel. Both follow the pattern of the earlier AnyLanguageModel, one familiar Swift API with a choice of model behind it, and both can pull open models from the Hugging Face Hub and run them on the device. The pitch is simple: language models are a small part of machine learning, and for jobs like transcription or routing a support ticket, purpose-built models are faster and cheaper than prompting a chatbot.\n\nThis matters to anyone building iOS or macOS apps with on-device AI, and to anyone tracking the growing family of \"decision models\" we have covered, including [Cloudflare's Clef-Omni](https://explainx.ai/blog/cloudflare-clef-omni-open-multimodal-decision-model-2026). The numbers in the announcement are the authors' own, so we flag what they measured and what they did not.\n\n## TL;DR: what you get\n\n| Question | Answer | \n|---|---|\n| What are the packages? | AnySpeechRecognizer (speech to text, speaker diarization) and AnyDecisionModel (typed decisions with probabilities) | \n| Who made them? | Mattt at Hugging Face, announced October 29, 2026 | \n| Which speech engines? | Apple's own, NVIDIA Parakeet (three variants), Alibaba Qwen3-ASR (0.6B and 1.7B) | \n| Which decision backends? | MLX, MLX decision heads (Clef), Core AI, llama.cpp, Jev, Ollama | \n| Requirements | Swift 6.2+, macOS 26 or iOS 26+, Apple silicon for local runtimes | \n| License of models | Mix of CC BY 4.0, Apache 2.0 and NVIDIA Open Model License, per the post's table | \n| Biggest caveat | Benchmarks from one Mac; not tested on iPhone; meetings remain hard | \n\n## AnySpeechRecognizer: keep Apple's API, change the engine\n\nApple introduced SpeechAnalyzer in macOS 26 and iOS 26: you add modules such as SpeechTranscriber and SpeechDetector to an analyzer, feed it audio, and read results from async sequences. The only engine behind it is Apple's. AnySpeechRecognizer mirrors SpeechAnalyzer, SpeechTranscriber, DictationTranscriber, SpeechDetector and AssetInventory, so for code that uses the shared API, the post says changing the import from `Speech` to `AnySpeechRecognizer` is enough. To use Parakeet you change only how the transcriber is created, passing a model object, and the steps for installing assets, running the analyzer and reading results stay the same.\n\n### The engines\n\n- **Parakeet v3** is NVIDIA's Parakeet TDT 0.6B v3: 25 European languages with automatic language detection.\n- **Parakeet Ultra** is Moondream's post-trained take on v3, which the authors found more accurate in most of their measurements.\n- **Parakeet Unified** is NVIDIA's English-only Parakeet.\n- **Qwen3-ASR** (0.6B and 1.7B) covers 30 languages and 22 Chinese dialects, including Mandarin, Cantonese, Japanese and Korean, which Parakeet does not. It returns one final result per segment of up to 40 seconds, with no live partial results, word timing or confidence, and it runs on the GPU more slowly than Parakeet, so it suits files better than live input.\n\nCore ML conversions are published on the Hub under the `mattt` account, each recording its source checkpoint and a checksum for every file, with the package pinning a fixed revision. Licenses carry over from the originals: CC BY 4.0 for Parakeet v3, Parakeet Ultra and Sortformer v2, the NVIDIA Open Model License for Parakeet Unified and Sortformer v2.1, and Apache 2.0 for Qwen3-ASR. Check these before shipping a commercial app.\n\n### Speaker diarization, which Apple does not offer\n\nApple's Speech framework has no diarization module, so the package adds a `SpeakerDiarizer` running NVIDIA's Streaming Sortformer on Core ML alongside an Apple or Parakeet transcriber on the same timeline. It tracks up to four speakers, including when they talk over each other, and a `SpeakerTimeline` attributes each timed word to a speaker so you can print a transcript by turn. Sortformer v2 is the default; v2.1, trained on more meeting data, was more accurate on meetings in the authors' tests.\n\nA laptop drawing with a small chip on its base, picturing Swift speech recognition running locally on Apple silicon\n\n### How well does it work?\n\nThe authors measured with their `hear` command-line tool on an M4 Max Mac Studio running macOS 27.0.1. Word error rates on LibriSpeech files of 3 to 8 minutes:\n\n| Transcriber | test-clean | test-other | \n|---|---|---|\n| Parakeet v3, file | 1.55% | 2.78% | \n| Parakeet v3, live | 4.73% | 4.29% | \n| Parakeet Ultra, file | 1.66% | 2.55% | \n| Parakeet Ultra, live | 2.65% | 3.28% | \n| Parakeet Unified, file | 1.43% | 2.19% | \n| Parakeet Unified, live | 1.45% | 2.42% | \n| Apple's transcriber, file | 2.12% | 3.97% | \n\nOn speed, they report Parakeet transcribing an hour of audio in about 20 to 40 seconds from a file, or roughly 1.5 to 2.5 minutes live, with about 100 MB of memory once Core ML has compiled the model. Qwen3-ASR is much slower and needs a few gigabytes.\n\nTwo honest limits from the authors: these come from one machine and a few datasets, and meetings are hard for everyone. On AMI recordings, where people interrupt each other, every engine tested had a word error rate above 23%. LibriSpeech is read book audio, so it flatters every system compared with a noisy call. Our coverage of other open speech models, such as [Granite Speech 5.0](https://explainx.ai/blog/ibm-granite-speech-5-0-2b-apache-open-asr-leaderboard-turbo-ctc-multilingual-diarization-2026) and [NVIDIA's Nemotron 3.5 Transcribe](https://explainx.ai/blog/nemotron-3-5-transcribe-open-30b-speech-model-granary-v2-40-languages-2026), shows how much the ranking shifts by dataset, so run `hear evaluate` on your own files.\n\n## AnyDecisionModel: probabilities instead of parsed text\n\nApps often ask a language model to decide: is this message urgent, which team should handle this ticket, how frustrated is this customer. The usual approach is to prompt and parse generated text, which yields one answer with no view of the alternatives. AnyDecisionModel instead derives each answer from the model's probabilities, returning typed Swift values.\n\n``` js\nenum Department: String, Choosable { case billing, technical, sales }\n\nlet model = MLXDecisionModel(modelID: \"mlx-community/Qwen3-4B-Instruct-2507-4bit\")\nlet session = DecisionSession(model: model, state: .text(ticket))\n\nlet urgent = try await session.probability(of: \"Does this convey urgency?\")\nlet team = try await session.choice(\"Which team?\", from: Department.self)\n```\n\nA `Choice` holds the selected case, a probability for each option and a confidence value; a `Score` does the same for rating levels. Because your enum conforms to `Choosable`, the result works in an exhaustive `switch`. A session holds a single state, text or JSON, and every question is about that state; questions are independent, and you can batch them with `decide(_:)`. For the MLX backend, the package builds a prompt with a one-token label for each allowed answer and reads the probability of each label from one forward pass.\n\nIf this idea sounds familiar, it is the same one behind the Jev family of single-token decision models; see our explainer on [how Jev works](https://explainx.ai/blog/how-does-jev-work-rlcd-system-one-model-explained-2026) and the [ranked decision models list](https://explainx.ai/blog/top-10-decision-models-jev-perplexity-openai-open-source-ranked-2026). The post is wry about it: decision models \"aren't new, exactly,\" but there is \"a lot of hype, and a crop of models trained to make decisions,\" so the authors wanted a Swift API ready for them.\n\n### The Emoji Magnet demo\n\nTo show the idea, the repository includes Emoji Magnet, a macOS app with 343 emoji. Type \"things you wear\" and matching emoji rise out of the pile; refine to \"things you wear in winter\" and non-matching ones fall back. There are no keyword rules: the app asks the model a yes-or-no question per emoji. With the hosted Jev backend, all 343 questions go in one request and settle in about half a second per phrase. Locally, Qwen3 4B on MLX answers 16 questions per forward pass, about 0.45 seconds per group on an M4 Max after loading, so a full pass over 343 emoji takes about 10 seconds.\n\nA small chip inside a handheld device cutaway, picturing on-device AnyDecisionModel inference\n\n### Trained decision heads: Cloudflare's Clef\n\nReading next-token probabilities works with general models, but they were not trained to decide. The post points to Cloudflare's Clef models, which add a trained decision head that takes a state and a schema of typed questions and returns probabilities for all options in one pass. AnyDecisionModel runs Clef-Flash, the 9B variant, via `MLXHeadDecisionModel`. The 4-bit model is a 6.2 GB download and its model card recommends at least 16 GB of memory. Clef can read images and video, but this backend accepts only text and JSON. For the broader Clef picture, see our [Clef-Omni post](https://explainx.ai/blog/cloudflare-clef-omni-open-multimodal-decision-model-2026) and the [comparison of Clairic, Jev, OpenAI Decisions, Clef and Microsoft Decision 1](https://explainx.ai/blog/clairic-vs-jev-vs-openai-decisions-api-vs-clef-vs-microsoft-decision-1-comparison-2026).\n\n### Calibration, and why confidence is not correctness\n\nRaw language-model probabilities tend to be overconfident, so the local backends accept a `Calibration`, which applies a temperature to every answer and a bias to yes-or-no answers; the post says to fit and evaluate on separate datasets. Models can also favor an option because of its position, so `rotationDebiasing` asks a choice question once per rotation of the options and averages, at the cost of one prompt evaluation per rotation.\n\nThe confidence value is based on the entropy of the probabilities. As the authors stress, it measures how concentrated the probabilities are, not whether the answer is right, so measure your own error rate before choosing a threshold. This lines up with an independent finding we covered: [LLMs repeat 96% of confident errors](https://explainx.ai/blog/llms-repeat-96-percent-jev-confident-errors-2026), which is a reminder that high confidence is not accuracy.\n\n### Six backends\n\nMLX runs a small model on Apple silicon; MLX decision heads run models with their own heads such as Clef; Core AI does the same as MLX on macOS 27 and iOS 27; llama.cpp reads decisions from a local GGUF model; Jev calls TypeSafe's hosted Jev through the System One API, or any server implementing it such as [Kev](https://explainx.ai/blog/kev-open-source-jev-clone-qwen35-family-2026); and Ollama calls a System One decision model locally or in Ollama's cloud (version 0.35.0 or later, or 0.40.5 or later for cloud models). The next-token backends label answers with yes/no, A to Z or 0 to 9, so a choice has at most 26 options and a score at most 10 levels; decision heads have no such limit.\n\n## What this means for what you build\n\n- **Replace brittle prompt-and-parse code.** If your app classifies text, a typed enum with probabilities is easier to test and route on than free text.\n- **Choose speech engines per job.** Parakeet for fast live English or European languages, Qwen3-ASR for Asian languages and file transcription, Apple's engine where you want zero extra download.\n- **Plan storage.** Downloads run from about 250 MB to 6.2 GB, only when your app asks for them, and go into the standard Hugging Face cache so a Mac app outside the sandbox shares files with Python tools.\n- **Calibrate before trusting.** Treat the confidence number as a signal, not a guarantee.\n\n## Setup and caveats\n\nAdd the packages to `Package.swift`:\n\n```\n.package(url: \"https://github.com/mattt/AnySpeechRecognizer\", from: \"0.1.0\"),\n.package(url: \"https://github.com/mattt/AnyDecisionModel\", from: \"0.6.0\", traits: [\"MLX\"]),\n```\n\nBoth need Swift 6.2 or later and macOS 26 or iOS 26 or later; the MLX build tasks use Xcode's Swift 6.4 or later and the Metal toolchain, and Core AI needs Xcode 27 and an OS 27 runtime. Without traits the decision package has no third-party dependencies and builds on Linux with the Jev and Ollama backends. One known conflict: AnyDecisionModel's CoreAI trait cannot be combined with AnySpeechRecognizer for now, because coreai-kit 0.7.3 requires swift-huggingface below 1.0 while AnySpeechRecognizer requires 1.x. Version numbers (0.1.0 and 0.6.0) signal early software, and the authors say they tested only on macOS 27, not iPhone.\n\n## What is claimed and what is verified\n\nThe package features and requirements are documented in the post and repositories, so you can check them yourself. The accuracy and latency figures are the authors' measurements on one M4 Max Mac Studio; they have published a `natural-speech-eval.py` tool to reproduce them, but we have not rerun it. The comparison with Apple's transcriber is on LibriSpeech only. Treat these as a starting point.\n\n## Related reading\n\n*Versions and measurements are accurate as of October 29, 2026.*", "url": "https://wpnews.pro/news/anyspeechrecognizer-and-anydecisionmodel-open-speech-and-decision-models-for", "canonical_source": "https://explainx.ai/blog/anyspeechrecognizer-anydecisionmodel-swift-parakeet-qwen3-asr-sortformer-clef-apple-platforms-2026?utm_source=rss&utm_medium=feed&utm_campaign=blog", "published_at": "2026-10-29 00:00:00+00:00", "updated_at": "2026-10-29 16:17:48.940274+00:00", "lang": "en", "topics": ["ai-tools", "developer-tools", "large-language-models", "ai-agents"], "entities": ["Hugging Face", "Mattt", "AnySpeechRecognizer", "AnyDecisionModel", "Apple", "NVIDIA", "Qwen3-ASR", "Sortformer"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/anyspeechrecognizer-and-anydecisionmodel-open-speech-and-decision-models-for", "markdown": "https://wpnews.pro/news/anyspeechrecognizer-and-anydecisionmodel-open-speech-and-decision-models-for.md", "text": "https://wpnews.pro/news/anyspeechrecognizer-and-anydecisionmodel-open-speech-and-decision-models-for.txt", "jsonld": "https://wpnews.pro/news/anyspeechrecognizer-and-anydecisionmodel-open-speech-and-decision-models-for.jsonld"}}