AIArticle
OpenBMB's Apache-licensed 2B model speaks 30 languages from raw text, with no phoneme frontend, in 8GB of VRAM.
Every notable open text-to-speech system of the past three years leans on the same trick: compress audio into discrete tokens with a neural codec, then train a language model to predict them. Microsoft's VALL-E made the recipe famous, and Bark, XTTS, and Alibaba's CosyVoice line are all variations on it. The appeal was obvious. Once speech looks like tokens, TTS becomes language modeling, and everything the field learned about scaling LLMs transfers over.
VoxCPM2, the 2B-parameter model OpenBMB shipped this April, is the strongest open-weights argument yet that the trick has outlived its usefulness. It generates speech directly in a continuous latent space, with no discrete audio tokens anywhere in the pipeline, and it covers 30 languages from raw text without a phoneme frontend or even a language tag. The weights are Apache-2.0, inference fits in about 8GB of VRAM, and the repo has passed 37,000 GitHub stars. Tokenizer-free TTS has stopped being a research position. For developers self-hosting speech synthesis, I'd now call it the default.
Why discrete tokens won, and what they cost #
Quantization was never free. A codec that squeezes speech into a few thousand codes throws away exactly the material that makes a voice sound alive: micro-prosody, breath, the difference between a flat reading and a pointed one. Codec-token systems also inherit a cascade problem. The tokenizer is trained separately from the model that consumes it, so errors compound across a boundary neither side can see past.
The old alternative had the opposite flaw. The VoxCPM technical report puts the dilemma plainly: "discrete tokens ensure stability but sacrifice expressivity, while continuous signals retain acoustic richness but suffer from error accumulation."
OpenBMB's answer is a hierarchy trained end-to-end on a MiniCPM-4 backbone. A text-semantic language model plans what to say and how it should sound, a residual acoustic model fills in fine-grained detail, and a local diffusion decoder renders continuous latents that a VAE turns into 48kHz audio. A differentiable quantization bottleneck between the stages gives the model the stability discrete tokens used to provide, without the hard boundary. None of the individual ideas is brand new. Microsoft's MELLE paper argued for autoregressive TTS without vector quantization back in 2024, and diffusion heads on LM backbones are the same move image generation made years ago. What OpenBMB did is ship the combination at production quality, at 2B parameters, under a license you can build a business on.
The multilingual claim is where the architecture pays off most for working developers. Classic pipelines like Piper depend on espeak-ng to turn text into phonemes, which means every new language is a frontend engineering project, and languages with messy orthography stay second-class. VoxCPM2 reads raw text. The per-language work moved out of your codebase and into OpenBMB's training corpus, which the model card says now exceeds 2 million hours across those 30 languages plus nine Chinese dialects.
What it takes to run #
Getting a first waveform out is a pip install:
from voxcpm import VoxCPM
model = VoxCPM.from_pretrained("openbmb/VoxCPM2")
wav = model.generate(
text="The deploy finished. Nobody knows why it worked.",
cfg_value=2.0,
seed=42,
)
On an RTX 4090 the repo reports a real-time factor around 0.30, dropping to roughly 0.13 with the project's Nano-vLLM streaming path, comfortably faster than playback. For serving, there's a vLLM-Omni integration with an OpenAI-compatible API, and a llama.cpp-omni build for CPU and Metal when you'd rather skip Python entirely. Cloning takes a short reference clip; voice design takes only a text description of the speaker you want, which is the feature I'd expect product teams to reach for first, since it sidesteps consent questions that come with cloning a real person.
Slot it into the stack and the comparison writes itself. XTTS is effectively unmaintained since Coqui shut down in early 2024, and its license bars commercial use anyway. Piper is still the right call on a Raspberry Pi, but its voices sound like 2021. And if you're paying a hosted API per character for narration, agents, or accessibility features at steady volume, an Apache-licensed model on a single 8GB GPU changes that math quickly.
The catches #
The benchmark numbers are OpenBMB's own. On the English side of Seed-TTS-eval the repo reports 1.84% word error rate and 75.3% speaker similarity, and the same table shows Alibaba's Qwen3-TTS ahead at 1.23%. So even by the vendor's accounting this isn't the top of the leaderboard, and no independent evaluation of VoxCPM2 has been published yet. Treat the multilingual intelligibility figures the same way.
The model card is candid about behavior, too: output varies across runs (diffusion sampling is stochastic, and seeds only pin down so much), quality tracks how much training data a language got, and very long inputs can get unstable. If you need deterministic, repeatable audio for regression-tested products, that's a real integration cost, not a footnote.
And cloning governance is entirely on you. The card prohibits impersonation and fraud, but Apache-2.0 carries no enforcement mechanism. If you deploy this behind a user-facing feature, the consent flow, watermarking, and abuse handling are your problem from day one.
Where this leaves the stack #
Direction of travel first: continuous-latent generation is winning, and I expect the codec-token generation of open TTS to age the way tacotron-era mel-spectrogram pipelines did. The tokens were scaffolding we needed while speech had to look like text to benefit from transformers. It no longer does.
Practically, VoxCPM2 is the model I'd start with today for self-hosted multilingual synthesis, with two watch items: hold the vendor benchmarks loosely until someone independent reruns them, and prototype your longest inputs early, because that's where the model card itself says the cracks show.
Sources & further reading #
-
[OpenBMB/VoxCPM](https://github.com/OpenBMB/VoxCPM) — github.com -
[openbmb/VoxCPM2 model card](https://huggingface.co/openbmb/VoxCPM2) — huggingface.co -
[VoxCPM: Tokenizer-Free TTS for Context-Aware Speech Generation and True-to-Life Voice Cloning](https://arxiv.org/abs/2509.24650) — arxiv.org
Mariana Souza· Senior Editor
Mariana covers the fast-moving world of machine learning and generative AI, with a particular focus on how these technologies are reshaping development workflows. When she isn't stress-testing the latest foundation models, she's usually at a local hackathon.
Discussion 0 #
No comments yet
Be the first to weigh in.