{"slug": "train-your-own-custom-tiny-tts-model-from-scratch-overnight-on-consumer-hardware", "title": "Train your own custom Tiny TTS model from scratch overnight on consumer hardware", "summary": "Developer Francisco Carlos Erra released TTSLibre v0.1.2, a 19.8M-parameter text-to-speech model trained on 177 hours of LibriTTS-R data, claiming it runs on consumer hardware with under 100ms latency and is fully reproducible under CC0/MIT licenses. The project aims to create a model under 100M parameters that outperforms Supertonic 3, with plans for a voice tool and multilingual support.", "body_md": "**Let's train the FOSS Supertonic / Kokoro successor.**\n\nA text-to-speech model under 100M parameters that answers instantly on a laptop CPU, sounds at least as good as Supertonic 3 and more expressive, and ships with everything needed to reproduce it.\n\n## simplescreenrecorder-2026-09-07_06.04.17.mp4\n\n**Current checkpoint: v0.1.2** (English PoC, 19.8M params, 177 h LibriTTS-R, whole-val Whisper WER 0.23): [FranciscoCarlosErra/ttslibre-poc-en](https://huggingface.co/FranciscoCarlosErra/ttslibre-poc-en) — cold-start and in-memory inference instructions in the model card. Rough, as promised in the plan.\n\n|  | Kokoro-82M | Supertonic 3 | **TTSLibre** | \n|---|---|---|---|\n| Weights | Apache 2.0 | OpenRAIL-M, use restrictions | **CC0, public domain** | \n| Training code | not released | not released | **MIT, in this repo** | \n| Training data | undisclosed, partly synthetic from closed TTS | undisclosed | **CC0, published with per-clip provenance** | \n| Voice tool | none official | none official | **MIT, style vector editor** | \n| Params | 82M | ~99M | **under 100M** | \n| Runs on | CPU | CPU | **CPU, and GPU under 1 GB** | \n\nFour artifacts, all free, all reproducible: **code, data, weights, voice tool.** You can retrain it, fine-tune it, embed it in a product, or fork the dataset, without asking anyone. That is the whole proposition.\n\nThe voice tool already exists for the two reference models. Both were built by reverse-engineering their style spaces without any training code, and both run on CPU.\n\n- [supertonic-voicelab](https://github.com/franciscocarloserra/supertonic-voicelab) : 40 new voice packs for Supertonic 3, sliders with measured safe ranges.\n- [kokoro-voicelab](https://github.com/franciscocarloserra/kokoro-voicelab) : same for Kokoro-82M, plus 257 per-phoneme knobs.\n\nWhat those tools could not do is the reason this project exists: they hit the ceiling of what you can do to a model without its training recipe. Retraining a style encoder, adding a language, fixing a bad phoneme, all blocked.\n\n- **Microscopic.** Under 100M parameters, under 1 GB in memory. Small enough to read the training script in an afternoon and run it on one consumer GPU.\n- **Instant.** First audio in well under 100 ms on a laptop CPU. No GPU required to be usable.\n- **Better than Supertonic 3.** That is the floor, not the goal: at least equal intelligibility and speaker similarity, and more expressive prosody. Measured, published, side by side.\n- **Free all the way down.** No clause anywhere in the stack that a lawyer has to read twice.\n\n- A TTS you can embed in any product without asking anyone.\n- A dataset you can train your own on.\n- A training recipe small enough to read in an afternoon and run on one GPU.\n- A voice tool: dial in a new voice from a style vector, no retraining.\n- Spanish and other languages as first-class citizens, not afterthoughts.\n\n| Requirement | Target | \n|---|---|\n| Parameters | under 100M | \n| Runtime | CPU and GPU via ONNX | \n| Latency | first audio well under 100 ms per sentence on a laptop CPU | \n| Memory | under 1 GB VRAM on GPU, under 1 GB RAM on CPU | \n| Voices | style vector, new voices without retraining | \n| Licenses | MIT code, CC0 data and weights. The most permissive there is. | \n\nTargets are measured, not claimed. Every tunable value lives in a config file, never in code.\n\nState of the art. Voice cloning from seconds of audio. Emotion control. Anything that needs a GPU to be usable. If some of it comes for free, fine.\n\n1. **Data.** Licensing rules, pipeline, first 100 clean hours of English.\n2. **Bake-off.** Two or three candidate architectures on those 100 hours. One metric. Results public. Winner sets the architecture.\n3. **First checkpoint.** Full English run. It will be rough. Ship it anyway. → v0.1.2 shipped (experiment 012).\n4. **Voice tool.** Style-vector editor on top of the checkpoint.\n5. **Languages.** Spanish first, then whatever contributors bring.\n\nWorst case this ends as a distillation of Kokoro and Supertonic with a public recipe, and that is still something nobody has published. Best case is a fully human-licensed dataset and a model anyone can ship.\n\nThe bake-off starts from what already works. Both reference implementations are vendored as submodules under `docs/references/`.\n\n- **StyleTTS 2 + ISTFTNet** (Kokoro's lineage). Training code exists and is public. Phoneme input, style diffusion optional, fast ISTFT vocoder.\n- **Flow matching over a compressed latent** (Supertonic's lineage). Paper published, no training code. Character input, no G2P, no aligner, ConvNeXt blocks.\n\n- **Data:** record, curate, align, verify licenses. Rules in`docs/COORDINATION.md` .\n- **Architecture:** propose a bake-off candidate as an issue. Bring a config and a reason.\n- **Compute:** a 3090/4090-class GPU for ablations. This does not need a swarm.\n- **Sponsors:** a few thousand dollars of cloud GPU for the main run.\n\n- `AGENTS.md` operating brief for humans and agents.\n- `docs/` sourced facts:`ARCHITECTURES.md` ,`DATASETS.md` ,`REFERENCES.md` ,`COORDINATION.md` .\n- `docs/references/` upstream implementations as submodules (papers linked in`docs/REFERENCES.md` ).\n- `experiments/NNN-*/` one directory per experiment, each with its README, config and result.\n- Training data is not in git. It is distributed via Hugging Face (`ttslibre` ); license audit in`docs/LICENSES.md` .\n\nPre-alpha. The Supertonic-lineage prototype in `experiments/` trains end to end (latent autoencoder, text-to-latent flow matching, ONNX export, Whisper WER in TensorBoard). First alignment reached on 2026-09-05: a 4-utterance overfit reaches WER 0.15 in 1000 steps on 1.4 GiB VRAM (`experiments/002-overfit-4-sentences/RESULTS.md`). Nothing generalizes yet.\n\nClone with `--recurse-submodules` to get the references.", "url": "https://wpnews.pro/news/train-your-own-custom-tiny-tts-model-from-scratch-overnight-on-consumer-hardware", "canonical_source": "https://github.com/franciscocarloserra/ttslibre", "published_at": "2026-09-07 09:20:50+00:00", "updated_at": "2026-09-07 09:58:51.643108+00:00", "lang": "en", "topics": ["artificial-intelligence", "machine-learning", "generative-ai", "ai-research", "ai-products"], "entities": ["Francisco Carlos Erra", "TTSLibre", "Supertonic 3", "Kokoro-82M", "LibriTTS-R", "Hugging Face"], "alternates": {"html": "https://wpnews.pro/news/train-your-own-custom-tiny-tts-model-from-scratch-overnight-on-consumer-hardware", "markdown": "https://wpnews.pro/news/train-your-own-custom-tiny-tts-model-from-scratch-overnight-on-consumer-hardware.md", "text": "https://wpnews.pro/news/train-your-own-custom-tiny-tts-model-from-scratch-overnight-on-consumer-hardware.txt", "jsonld": "https://wpnews.pro/news/train-your-own-custom-tiny-tts-model-from-scratch-overnight-on-consumer-hardware.jsonld"}}