A from-scratch Vietnamese TTS hits 48 kHz audio, instant voice cloning, and real-time CPU inference with an OpenAI-compatible streaming API.
- VieNeu-TTS v3 Turbo ships 48 kHz Vietnamese and English TTS with instant voice cloning, open-sourced under Apache 2.0 on GitHub
- Trained from scratch on 10,000+ hours of bilingual audio with 25 built-in preset voices across Northern, Central, Southern dialects
- Real-time CPU inference via torch-free ONNX install; RTF 0.5 on a 12th-gen Core i5, 0.35 with int8 quantization
- GPU streaming on RTX 3060 hits 115 ms first-audio and 16 concurrent streams via a single-replay CUDA graph
- OpenAI-compatible
/v1/audio/speechendpoint drops into Pipecat, LiveKit, and the OpenAI SDK with Docker profiles for CPU and GPU - Proprietary v4 with near-original cloning fidelity is held back from open source over misuse concerns; v3 Turbo remains the latest open release
Vietnamese text-to-speech has fewer capable open models than English, and many available systems pair synthetic-sounding output with demanding hardware. The VieNeu-TTS project, now above 3,000 GitHub stars, has released v3 Turbo with 48 kHz output, voice cloning from a clip lasting 3 to 8 seconds, and an OpenAI-compatible streaming API. The current open release uses the Apache 2.0 license.
According to the project documentation, its bilingual architecture was trained from scratch on more than 10,000 hours of Vietnamese and English audio. Reported tests put time to first audio at about 115 milliseconds, batched throughput at 50 to 90 times real-time on an RTX 3060, and CPU generation at roughly 1.6 to 1.8 times real-time. Those characteristics support local voice agents, offline applications, and deployments that keep speech data away from cloud services.
What v3 Turbo ships #
The model combines the MOSS audio codec, which represents speech as model-friendly audio tokens, with a custom phonemizer called sea-g2p that maps written text to pronunciation units.
- 48 kHz audio output, increased from 24 kHz in earlier versions
- 25 preset voices, including 10 labeled “Editor’s Picks” for stable generation without a reference clip
- Experimental emotion and non-verbal cues embedded in text, including
[cười]for a chuckle,[thở dài]for a sigh, and[hắng giọng]for clearing the throat - Batched generation with up to 32 items per batch
- A multi-speaker
Conversationmode that batches an entire script across speakers - Voice cloning from a 3- to 8-second recording, with automatic denoising of the reference audio
- Low-rank adaptation, or LoRA, for memory-efficient fine-tuning of a voice or reading style on a consumer GPU with as little as 6 GB of VRAM
CPU inference without PyTorch #
VieNeu-TTS keeps its minimal CPU installation free of PyTorch by running through ONNX Runtime and avoiding any Torch import. On a CUDA host, the same API automatically switches to PyTorch and enables batched inference. Installing
This story is for Pro members #
You've reached the end of the free preview. Upgrade to AlphaSignal Pro to read the full article - and everything else behind the paywall.