Text-to-speech is basically solved for reading text out loud. Turning a story into something that sounds like a radio drama is not. Once you try it, you notice that the voice model is the easy part. The hard part is everything around it: working out who says each line, keeping every character's voice the same across scenes, directing the delivery, and mixing dialogue with ambience and music so the result doesn't sound like a screen reader.
ShengChang (声场) is my attempt at that whole job. It's a personal AI radio-drama workbench: import a novel chapter as a .txt file, get a directed, mixed episode. It runs on your own machine, it's MIT-licensed, and each character keeps one voice from the first scene to the last.
The reason is a bit personal. I'm a heavy audiobook fan, and at one point I wanted to do voice acting on the side. My own voice was never going to get me there, so the idea just sat. Over the last couple of years, voice models got good enough to clone a voice and take direction on emotion and pacing, and it clicked: if I can't perform the parts, maybe I can direct them.
The name comes from acoustics. 声场 (shēngchǎng) means "sound field": how sound fills a space. In a good radio drama you can close your eyes and still tell who's standing where, how far away they are, and whether you're in a forest or a palace hall. That's the job I want this tool to do. Its own tagline puts it as 「让故事，有声有场」, roughly "give a story its voices, and the space they live in."
The pipeline looks like this:
scene/line breakdown → per-character voice mapping → directed TTS → FFmpeg mix desk with sidechain ducking → loudness-normalized MP3
The stack is plain on purpose: React + Vite for the desktop-first UI, Node.js + Express on the server, the built-in node:sqlite module for storage, and FFmpeg/ffprobe for rendering. One thing to know up front: the UI and the sample content are Chinese only, and English support isn't on my plan yet.
It's for:
One-click episode. One button runs five checkpointed stages: breakdown → preset voice mapping → batch generation → audio health check → episode export. Before it starts, you get an estimate (line count, characters, predicted cache hits). If a stage fails, you retry from that checkpoint. Re-runs only re-voice the lines you actually changed (text, emotion, speed, or locked voice). Everything else is reused.
Direction, not just text. Every line can carry subtext, emotion, intensity, speed, and distance. ShengChang folds those into one voice instruction for the engine. For Breeze, ShengChang writes the instruction itself (emotion level + speed + subtext, max 300 characters). If a line has no direction at all, it sends an empty instruction, so you get a plain voice clone.
Direction Lab. Pick a line and render it three ways side by side in the Inspector. Takes are cached by their direction parameters, so when you apply the one you like and regenerate, it's an instant cache hit.
Voices that stay put. With a local Breeze server, you can clone a voice from a 5–30 second reference clip (10–30 s recommended), or design one from a text description, audition it, and lock it.
A real mix desk. Each scene gets its own timeline. Dialogue is sequenced automatically when you don't set in-points, and background beds can loop. Music and ambience duck under dialogue through FFmpeg sidechain compression. The master chain normalizes to a target loudness (−16 LUFS by default) with a −1 dBTP ceiling, and exports 48 kHz stereo MP3 at 192 kbps.
Sound assets with provenance. The app ships with 20 CC0 assets (ambience, foley, music beds) and recommends matches from each sound event's description, all computed locally. You can also plug in an AI SFX service or import your own files. Each export comes with a source list that groups every asset as built-in, AI-generated, or imported.
Zero-key first run. A fresh install seeds a sample project: an adapted excerpt from chapter 27 of Journey to the West (public domain), condensed to 19 fully directed lines. It defaults to an offline demo engine, so the whole one-click pipeline runs end to end with no API key, no signup, and no cost. The idea is simple: the plumbing is real, and only the voice is a placeholder.
Requirements: Node.js 22.5+, with FFmpeg and ffprobe on your PATH.
npm install
npm run dev # workbench on http://127.0.0.1:4173
This starts the web UI on 127.0.0.1:4173 and a local model gateway on 127.0.0.1:4174. Then:
For real voices, run the bundled Breeze TTS service. It currently needs an Apple Silicon Mac:
cd breeze
uv venv --python 3.12 .venv
uv pip install --python .venv/bin/python -r requirements.txt
.venv/bin/hf download mlx-community/Breeze-TTS-2-mlx --local-dir models/Breeze-TTS-2-mlx
./start.sh
curl -s http://127.0.0.1:4180/health
Then open Settings, choose the local Breeze-TTS-2 engine, and set the address to http://127.0.0.1:4180. To import your own novel (a .txt file), first add any OpenAI-compatible text model (base URL, key, model name) and click 测试连接 ("Test connection"). The import button stays locked until that test passes.
You can also use Docker. The image includes Node 22 and FFmpeg, runs as a non-root user, and serves everything on one port, 4174:
docker build -t shengchang .
docker run -p 4174:4174 \
-v shengchang-runtime:/app/.runtime \
-v shengchang-config:/home/node/.shengchang \
shengchang
Loopback only. The server binds to 127.0.0.1, and /api rejects any non-loopback Host or Origin header, which blocks cross-site form posts and DNS rebinding. Credentials live in ~/.shengchang/config.json with mode 0600. The browser never sees a key in plaintext; the UI shows only the last three characters. If you change an endpoint, the saved key is not sent to the new address.
No default provider. The text model can be any OpenAI Chat Completions–compatible endpoint: a hosted API like DeepSeek or OpenAI, or a local server like Ollama, LM Studio, vLLM, or llama.cpp. You do need one to import your own novel, though: the current UI blocks the import until the connection test passes. (The server still has a rule-based breakdown fallback, but the current import flow doesn't reach it.)
No silent engine switching. If Breeze is unreachable, the UI tells you and offers a retry or a link to Settings. It never falls back to another engine, and never to a paid one. There's a dedicated test for this. I'd rather stop the pipeline than spend someone's money without asking.
Production plumbing. Breakdown, generation, and export all run as persistent, resumable tasks in SQLite with stage checkpoints, so unfinished work goes back in the queue after a restart. Audio is cached by content: the same model, voice, text, and direction always reuse the same file. When the cache passes its size limit (50 GB by default), the least recently used entries are evicted, but files still referenced by a project are never deleted. When you edit a line, its old audio is marked stale instead of being quietly removed.
Tests that exercise the real chain. Besides unit tests, npm run test:e2e starts the actual server and runs generation and the one-click pipeline end to end. The mix tests render real timelines through FFmpeg and check duration and format with ffprobe. Both suites run in GitHub Actions.
This is an early project, and I'd rather say so up front:
v0.1.0, the license is MIT, and CI is passing. breeze/requirements.txt was pinned from a working environment but hasn't been tested on a clean machine. The 8-bit Breeze weights should save memory, but their quality and stability are untested.
What's next isn't a long feature list. The focus is output quality: more accurate breakdowns, more natural delivery, and episodes that sound more like real radio drama.
If you want to hear a book you love as a radio drama, or you just want to see how a text-to-episode audio pipeline holds together, clone the repo and run the sample project. It takes no keys and a couple of commands.
👉 https://github.com/yanjiliang/shengchang-ai-studio
If it's useful, a star helps. Issues, Discussions, and PRs are all welcome.