Drama Studio: local realistic text-to-speech synthesis engine for Apple Silicon

Developer Peter Marelas released Drama Studio, a native macOS app that runs Fish Audio's 35B Drama 3 text-to-speech model locally on Apple silicon using hand-written Metal compute kernels for the mixture-of-experts backbone and codebook tower, with the DAC audio codec running through Core ML and no MLX or PyTorch at runtime. The app requires about 49 GB of free disk, maps roughly 41 GB of model weights (a 41.5 GB download pinned to revision e0a9218 plus a 1.5 GB Core ML voice codec), and recommends 64 GB or more of unified memory; generation was tested on an M3 Max with 128 GB, while an M1 Mac Mini with 16 GB failed twice with a GPU out-of-memory error. The model weights fall under the Fish Audio Community License, which the app requires users to accept before generating anything and which prohibits cloning a real person's voice without consent.

By Peter Marelas. A native macOS app that runs Fish Audio's Drama 3 https://huggingface.co/fishaudio/drama-3 text-to-speech model locally on Apple silicon. The 35B mixture-of-experts backbone and the codebook tower run on hand-written Metal compute kernels that read the released safetensors weights in place. The DAC audio codec runs through Core ML. No MLX, no PyTorch at runtime. Built with Fish Audio. The model weights are under the Fish Audio Community License https://huggingface.co/fishaudio/drama-3/blob/main/LICENSE ; cloning a real person's voice without their consent is not permitted. Drama-Studio-Promo-10MB.mp4 - Apple silicon Mac. Checked generation requires macOS 26 or later and an installed on-device speech model for the script language. Unchecked preview targets macOS 15. Intel Macs are not supported by the engine. - About 49 GB of free disk for the model, codec and setup caches. Allow more space for generated audio, saved projects and app updates. - 64 GB or more of unified memory is recommended. This is not a verified minimum. Generation has been tested on an M3 Max with 128 GB. An M1 Mac Mini with 16 GB failed twice with a GPU out-of-memory error before producing audio, despite paging. See the hardware test https://github.com/maralski/drama-studio/blob/main/docs/ui-verification.md 16-gib-mac-mini-hardware-test . 32 GB and 64 GB Macs remain untested. The engine maps about 41 GB of model weights. - A display that can fit a 900 by 640 point studio window. - An internet connection for the initial downloads and update checks. Speech generation runs locally after setup. The setup memory override tests the warning only. It does not simulate a lower-memory Mac or establish that generation works on one. 1. Download DramaStudio-<version .dmg from the latest release https://github.com/maralski/drama-studio/releases/latest . 2. Open the disk image and drag Drama Studio to Applications. 3. Open Drama Studio from Applications. The release is not notarized, so macOS blocks the first open. Open System Settings, go to Privacy & Security, scroll to Security, and click Open Anyway next to the message about Drama Studio. Confirm with your password. If you use Terminal, you can run xattr -dr com.apple.quarantine /Applications/DramaStudio.app instead. 4. Read the license terms in the setup window and click I Agree . Fish Audio licenses the model under the Fish Audio Community License, and Drama Studio generates nothing until you agree. 5. Wait for the downloads. The window shows the progress of each file, the total, and the time left: - the Drama 3 weights from Hugging Face, 41.5 GB, pinned to revision e0a9218 ; - the Core ML voice codec from the release, 1.5 GB. You can pause, or quit and open the app again. The download continues where it stopped. Each file is checked against a SHA-256 value that ships with the app before it is kept. 6. the Drama 3 weights from Hugging Face, 41.5 GB, pinned to revision 7. Wait 30 seconds to a minute while the first load tunes the GPU kernels for your Mac. 8. Click Generate to speak your first line. If you already downloaded the model with hf download fishaudio/drama-3 , Drama Studio uses that copy and skips the 41.5 GB download. If you are below 64 GB of memory, the setup window says so before anything downloads. You can continue at your own risk. Use Clear Script in the toolbar or Script menu to empty the editor. For an imported document, Clear Section empties only the selected section. Existing takes stay available and may need regeneration after the text changes. Use Edit Undo to restore cleared text. Clearing is disabled during generation. Screenplay Mode requires a speaker name, a colon and a space before dialogue: Alex: Hello there. Sam: Good morning. A line without a name continues the previous speaker. Text before the first named speaker belongs to Narrator. Up to 16 speakers are supported. If the script has no named dialogue, turn Screenplay Mode off for plain narration, or use the help button beside the error for the format and examples. Drama Studio checks GitHub for a newer stable release at launch and downloads it in the background. Automatic checks and downloads are on by default. Turn them off in Settings Updates , or use Drama Studio Check for Updates to check manually at any time. Once the download is verified, Settings Updates offers Install and Restart . Save your project and finish generation or API requests before confirming. Later leaves the running app unchanged; click Review Update in Settings Updates when ready. Check for Updates opens the same tab. There is no separate Software Update window. Quitting normally does not install an update. A deferred update is downloaded again on the next automatic or manual check. Updates require the app to be installed in a folder you can write to, such as Applications or your user's Applications folder. Apps running from a mounted DMG, App Translocation, or a Swift build directory cannot update themselves. The updater changes only the app bundle; models, voices, and projects stay in place. It uses the release's DramaStudio-<version .dmg and SHA256SUMS , checks the bundle identity, version, and minimum macOS requirement, verifies its code signature, and requires the same signing identity for Developer ID builds. Ad-hoc builds rely on GitHub HTTPS and the published checksum for authenticity. After restart, setup compares the installed codec's recorded archive SHA-256 with the digest shipped in the updated app. Matching codecs are reused even when the release URL changes. A changed digest or missing codec models triggers a verified download and installation. Older installs without a recorded digest need one verified download to establish it. - Script editor with import of .txt , .md , .rtf , .rtfd , .html , .docx , .srt and .vtt , drag and drop, and a screenplay mode that turns Name: line dialogue into Drama 3 speaker tags up to 16 speakers . - Takes : Generate or Generate All produces a fresh take each time. A scrolling grid shows take durations, notes, export choices and outdated audio. Other controls include an Expressiveness slider under Advanced one control over temperature, top-p and top-k, with the vendor's defaults in the middle and the Hugging Face Space's sampling at the top , and an insert menu for performance cues such as laughs , sighs and <whisper …</whisper . Each chunk's frame cap follows the length of its text. Normal generation checks each decoded sentence with local, audio-only speech recognition before playback or reuse as history. It also checks each complete adjacent pair before releasing the next sentence, so a bad join retries only the unplayed sentence within the same attempt limit. Each raw sentence must pass both recognizers with the existing native confidence checks. For a pair of those exact verified sentences, the native checker may mark only the first nonterminal word of the second sentence uncertain while both full transcripts still match. That narrow boundary case uses the raw sentence's confidence; ambiguous word positions, missing confidence, or other uncertain words fail. Arbitrary-file checks have no such generated-sentence evidence. These bounded windows cover every generated join within one request; they do not establish a global transcript for a long recording or certify joins between separate requests. Missing or uncertain words trigger a bounded retry; exhaustion fails instead of creating a completed take. Checked generation requires macOS 26, an installed Apple on-device speech model for the script's language, and the bundled independent recognizer. Both audio-only recognizers must find every scripted word before a sentence is accepted. A disagreement retries generation; an unavailable checker stops generation. These checks reduce word errors but cannot guarantee recognition accuracy or acting quality. Older takes without word-check history are kept for playback and export, but do not automatically become voice references or retake history. Voice auditions always use checked generation. Recognition checks word coverage, not acting quality or perfect cue execution. - Document narration : a PowerPoint deck, a PDF or an EPUB opens as sections, one per slide, page or chapter, and narrates section by section with the same voice throughout. Exports are the whole narration, one file per section with a manifest.csv , SRT or WebVTT captions, and for a deck a slide narration package. Projects save and reopen as .dramaproj bundles see Documents documents . - Pronunciation rules in Settings rewrite words before the model hears them, in the app and through the API. - Voice cloning from one or more reference clips any format AVFoundation reads plus their transcripts, kept in a local voice library. To reuse a generated voice, select a take and click Save Voice to Library beside Use for Export . Name the voice and save it. The app selects it for future generations, including after you edit the script. This works with Model default takes and individual document sections. Complete, checked takes containing one speaker can be saved; outdated script text does not prevent it. Save Session Voice in Cast saves the retained session voice instead. - Streaming playback as chunks finish, scrubbing, a speed control from 0.75x to 1.5x, the sentence being spoken highlighted in the editor, and export to WAV or M4A. - A local HTTP API and an MCP server , so scripts and agents can speak through the same engine and voice library see Agents and programs agents-and-programs . - Menu bar residency : closing the window leaves Drama Studio in the menu bar the waveform-and-microphone icon with the API still serving. The menu shows the model state, the API address, and the current job. ⌘Q quits. The system requirements system-requirements also apply to source builds. Install the following tools and model files: - macOS 15 or newer, Xcode 26 command line tools. - The model in the Hugging Face cache: uvx --from huggingface hub hf download fishaudio/drama-3 - The Core ML codec models, built once from codec.pth : cd tools && uv venv --python 3.12 .venv && uv pip install --python .venv/bin/python "torch==2.7.1" coremltools "numpy<2.4" einops soundfile .venv/bin/python -I ../Sources/DramaEngine/CodecConversion/convert codec.py They land in ~/Library/Application Support/DramaStudio/Codec/ encoder 809 MB in fp32, decoder 353 MB in fp16 . python3 tools/bootstrap word recognition.py downloads and checks pinned framework and local speech models swift build -c release python3 tools/stage word recognition.py --bin .build/release tools/make app.sh assembles dist/DramaStudio.app, with drama-mcp and drama-say in Contents/MacOS open dist/DramaStudio.app Command line: The app build stages the recognition runtime automatically. CLI builds and tests need the bootstrap and staging steps above. The English and multilingual models use about 975 MB on disk; recognition runs locally with no script hints. English uses small.en ; other supported languages use small . Only the selected model is loaded. Recognition requires both an installed Apple model and a supported independent model language. Older native-only checks remain recorded in project files, but their audio is excluded from automatic checked-reference reuse. Run the retained wrong-word and complete-ending controls with swift test -c release --filter LocalWordVerifierTests . On macOS 26 these controls fail if required local assets are missing; they do not skip that failure. tools/make cli.sh assembles a signed, relocatable CLI runtime in dist/drama-cli , including the helper, framework, resource bundle, models and licenses. Set DS VERIFY NATIVE WORD CHECKS=1 when running either packaging script to require the installed native model and exercise the full dual check on the retained complete-ending and wrong-word recordings. Packaging always tests the independent runtime. The full offline check can also be rerun with drama check-words <WAV <script <accepted|rejected , or with the app executable's --check-words flag. These commands do not load the speech generation model. .build/release/drama tune once per Mac, about 30 s; the app does it on first load .build/release/drama generate --text "Hello there." --out frames.npy .build/release/drama bench add --untuned to compare against the hand-written schedule .build/release/drama serve the HTTP API without the app; --port 8780, --no-token .build/release/drama-say "Hello there." speaks through the running app, like say; see From the terminal DRAMA PREVIEW ENGINE=1 runs the app or drama serve on a mock engine that produces placeholder tones, for working on everything but the model. The app looks for the weights in this order: the folder chosen in Settings, its own download in ~/Library/Application Support/DramaStudio/Models/drama-3-e0a9218/ , then the newest Hugging Face cache snapshot. A build from tools/make app.sh has no codec download, so its setup window shows the Python conversion command above when the codec models are missing; tools/make release.sh builds the release DMG and the codec zip and writes the zip's URL and SHA-256 into the app. DRAMA STUDIO HOME=<folder moves the downloads, the codec, the tuning and the settings into one folder and ignores the Hugging Face cache, to try a first run without touching your own. DRAMA STUDIO MEMORY GB=16 shows the low-memory warning on a Mac that has enough. tools/pin model plan.py --check compares the pinned download plan with Hugging Face. Importing a .pptx , .pdf or .epub File Import Script or Document, drag and drop, or opening the file with the app opens it as sections: a deck's slides, with the speaker notes as the text and the slide text when a slide has no notes; a PDF's pages; an EPUB's chapters. The list shows each section's label, a preview, its duration once narrated, a skip box and a speaker picker. The editor edits the selected section. - Generate narrates every section that is not skipped, in order, one request per section, cast with that section's speaker's voice. The speaker's previous section goes along as history, and with no library voice cast the speaker's first narrated section is also the voice, the way takes and the live route already carry a voice. Narrate This Section generates one section again. Click a take to audition it, then click Use for Export to choose it for the narration. The Export badge identifies the chosen take; auditioning another take does not change it. - Editing text or changing a speaker or voice marks the affected audio as Needs regeneration . Generate Changed Sections updates only missing or outdated sections. Export asks before writing outdated or incomplete audio. Earlier takes remain available, including after a project is reopened. - Export writes the whole narration, Per Section numbered files and a manifest.csv , Captions as SRT or WebVTT, timed from the frames each sentence took, and for a deck a Slide Narration Package: slide-NN.wav per narrated slide and a README.txt on adding them with Insert Audio. The app does not change the deck. - Save Project writes a .dramaproj bundle with the document or script, settings, voice assignments, session voices, and every take with its original text, voice assignments, codes, and audio. Library assignments still refer to voices in the local library. Save updates the current project; Save Project As creates a copy. Format 2 projects require this version of Drama Studio. Older projects still open, with a verification warning for audio whose generation history was not stored. - Import, Open Project, and Quit offer Save, Discard Changes, or Cancel when work is unsaved. A failed or cancelled save leaves the current session open. After editing pauses for two seconds, the app writes a recovery draft in the background; drafts wait until generation stops. The next launch offers to restore a draft left by an interrupted session. Recovery does not replace the last manually saved project. - Flatten to Script turns the sections into one plain script and removes their takes from the session. Its confirmation offers to save a project copy first. On the five-slide sample deck python-pptx , slide 3 without notes , the real engine at the default settings narrated 24.3 s of audio in 34.5 s including the model load. Whisper small.en heard slide 4, whose notes read "Revenue grew twelve percent. Costs stayed flat, and margins improved.", as "Revenue grew 12%. Cost stayed flat, and margins improved." Two slides lost words: slide 1 spoke only "Welcome, everyone." of its two sentences, and slide 3 only its title. Each was one engine chunk in that build the splitter joined a sentence under 24 bytes to the next , and with "Stream audio while generating" on, a chunk that comes out short is kept rather than generated again. With the setting off, Narrate This Section gave slide 1 both sentences and slide 3 "How it works Import". That is one run per slide, not a measurement. Because narration is exported, document sections now always generate in the retrying sentence mode whatever the streaming setting says; playback of a section starts when the section is done rather than three seconds in. eval/reliability/REPORT.md is a 330-run study of whether every sentence comes out as speech: 33 conditions short and long lines, numbers, abbreviations, all caps, performance cues, multi-speaker tags, Spanish, French and German, a cloned voice, both expressiveness extremes, and two control conditions , five seeds each, in full and balanced . | rows | full | balanced | |---|---|---| | plain prose under the shipped defaults | 130 of 130 | 126 of 130 | | performance-cue lines | 16 of 20 | 14 of 20 | | retries off control | 1 of 5 | 3 of 5 | | reference 300-byte chunking control | 7 of 10 | 9 of 10 | Paired by seed, balanced is never right where full is wrong, and wrong where full is right in 4 of 130 prose pairs. Performance cues are the one area where both modes drop content a few times in twenty; laughs can also turn into a laughter loop that the app's text-derived frame cap ends near 8 s. Reproduce with eval/reliability.py slices under eval/reliability/slices/ . The balanced mode was removed after this study, so the engine has one precision, full ; the column stays as the record of why. The cue dropouts were then investigated on eight cue lines, five seeds each, eleven variants eval/cues/ , section "Performance cues: investigation" of the report . The model stops at a cue that follows a sentence end, whatever the chunking; longer chunks lose more. Three guards now ship: a laughter loop is detected in the code stream the main tokens cycle with a period of 17 or more, which the 16-frame repetition redraw forces and the chunk is regenerated; a chunk that stops without eos is regenerated when it spoke under its expected duration; and the frame cap grows by 1.5 s per cue so a laugh or a pause is not cut off. Every sentence present went from 44 of 55 runs to 38 of 40 on the cue lines; on six prose lines at five seeds the old and new rules score 30 of 30 and 29 of 30, the one difference being a line that both rules retry and where the new rule's random path lost. The Space's sampling temperature 1.0, top-p 0.9, top-k off was then run over the whole study section "The Space's sampling on prose" : no prose row lost in 130, every sentence of the 20 cue lines, 5 retries instead of 19, the two-sentence line reaching the end of speech on its first attempt with retries off in 5 of 5, and 0.007 more word error rate on prose standard error 0.004 , mostly abbreviations and a few slurred words. It is the top of the Expressiveness slider and the API's expressiveness: 1 ; the vendor's sampling stays the default. The Insert Cue menu says what remains: about one take in twenty still drops the clause after a cue, so listen before exporting. While Drama Studio is open, including when only its menu bar icon is left, it serves an HTTP API on http://127.0.0.1:8780 . drama serve serves the same API without the app. The API listens on the loopback interface only. Settings holds the port, a switch for starting the server with the app, and the token. The menu bar has Start and Stop. When the port is taken, the menu shows "API: Port 8780 is already in use." Every route except /v1/health needs the bearer token from ~/Library/Application Support/DramaStudio/api-token , created the first time the server starts. The token keeps web pages, which can reach localhost, from driving the engine. drama serve --no-token turns it off. TOKEN=$ cat ~/Library/Application\ Support/DramaStudio/api-token AUTH="Authorization: Bearer $TOKEN" The speech request follows the SGLang Drama 3 cookbook https://sgl-project.github.io/sglang-omni/cookbook/fishaudio drama3.html : | Route | Does | |---|---| | GET /v1/health | {status, model loaded, engine: "metal" or "mock", busy, tuned, version} . tuned is null until the model loads. While a generation runs it adds estimate: {elapsed seconds, remaining seconds, frames done, frames expected} ; remaining seconds is null once the last sentence is in its silence tail or a retry runs. | | GET /v1/models | The one model, fishaudio/drama-3 , with its snapshot id. | | POST /v1/audio/speech | Speaks input . Returns the audio file, or a PCM stream. | | POST /v1/tts | The same, with the text in text . | | GET /v1/audio/speech/stream | Speaks the input query parameter as Server-Sent Events. See Streaming streaming-framing . | | POST /v1/audio/speech/live | Speaks text while the client is still sending it, and streams the audio back on the same connection. See Streaming streaming-framing . | | GET /v1/voices | The voice library: id, name, transcript, seconds, source file. | | POST /v1/voices | Clones a voice: JSON {name, transcript, audio base64 or audio path} , or multipart with a file part plus name and transcript fields. | | DELETE /v1/voices/{id} | Deletes a voice. | | GET /v1/takes/{id} | One of the last 20 takes: {id, input, text, seconds, frames, codes base64, previous take ids} . | The speech body takes these fields: | Field | Meaning | |---|---| | input | The text. Performance cues such as laughs and <\|speaker:1\| tags work as in the app. | | voice | A library voice, by id or name. | | references | Clips to clone on the fly: {audio path or audio base64, text} . | | speaker voices | Library voices per speaker for a multi-speaker script: {"0": "Narrator", "1": "<id "} , with no gaps from speaker 0. | | response format | wav default, 16-bit , pcm raw 16-bit little-endian , or m4a . | | stream | true streams completed, checked chunks. | | expressiveness | 0 to 1, default 0.5: the app's Expressiveness slider. 0.5 is the vendor's sampling, 1 the Hugging Face Space's. | | seed | A fixed sampler seed, for repeatable takes. | | character | A voice designed from a description instead of a clip: {name, description, scene?} . | | note | A director's note, such as "softer and warmer" . The line is generated again as one turn, with the note as a direction and as a cue before every sentence. | | previous takes | Earlier takes of the line, oldest first: {take id} or {text, codes base64} . The last two are context. With no voice or references , the first one is the voice. | | apply pronunciation | true default rewrites input with the pronunciation rules in ~/Library/Application Support/DramaStudio/pronunciation.json , the same rules the app uses, before it is spoken; false speaks it as sent. | A request takes at most one of voice , references , speaker voices , and character . Every generation is a take: the response carries its id in X-Drama-Take-Id , and the server keeps the last 20. A take id in previous takes also brings the takes that take was made after, so naming a line's latest take carries its first take, the voice. Leave out input to retake the line of the last named take, with its voice and character. codes base64 is the take's codes as 16-bit little-endian integers, codebook by codebook, as GET /v1/takes/{id} returns them. A file response carries X-Drama-Frames , X-Drama-Seconds , and X-Drama-Generation-Seconds . The engine runs one generation at a time. A request that arrives while another runs, through the API or in the window, gets 429 with Retry-After . A bad request gets 400 with a plain-text message that names the field. curl -sS localhost:8780/v1/health curl -sS -H "$AUTH" localhost:8780/v1/audio/speech \ -d '{"input": "Hello there. laughs Good to see you.", "voice": "Narrator"}' -o hello.wav curl -sS -H "$AUTH" localhost:8780/v1/tts -d '{"text": "The alias route."}' -o alias.wav curl -sS -H "$AUTH" localhost:8780/v1/audio/speech \ -d '{"input": "Cloned on the fly.", "references": {"audio path": "/path/to/clip.wav", "text": "What the clip says."} }' -o clone.wav curl -sS -H "$AUTH" -D headers.txt localhost:8780/v1/audio/speech \ -d '{"input": " low and menacing Pray ye never find out.", "character": {"name": "Captain", "description": "gravelly old male voice, weathered and slow"}}' -o captain.wav TAKE=$ awk 'tolower $1 == "x-drama-take-id:" {print $2}' headers.txt | tr -d '\r' curl -sS -H "$AUTH" localhost:8780/v1/audio/speech \ -d "{\"note\": \"softer and warmer\", \"previous takes\": {\"take id\": \"$TAKE\"} }" -o captain-retake.wav All three streaming routes send completed chunks when retries are enabled. The engine validates each attempt before releasing audio, so a rejected attempt can be replaced for any voice. The audio is always 16-bit little-endian mono at 44.1 kHz, as the X-Drama-Format: pcm s16le mono and X-Drama-Sample-Rate: 44100 headers say. stream: true and the event stream also send X-Drama-Take-Id and X-Drama-Estimated-Seconds before the first audio. PCM. stream: true on POST /v1/audio/speech answers with Transfer-Encoding: chunked and raw PCM in the HTTP chunks. A chunk boundary means nothing; concatenate the chunks. Every chunk holds a whole number of 2048-sample frames. Pipe it into a player: curl -sSN -H "$AUTH" localhost:8780/v1/audio/speech \ -d '{"input": "A long script, read as it is generated.", "stream": true, "response format": "pcm"}' | ffplay -f s16le -ar 44100 -ch layout mono -nodisp -autoexit - Server-Sent Events. GET /v1/audio/speech/stream takes its fields as query parameters: input required , voice , expressiveness , seed , and character name , character description and character scene for a designed voice. apply pronunciation=false turns off the pronunciation rules, as in the speech body. It answers with Content-Type: text/event-stream and these events: | Event | data | |---|---| | audio | Base64 of the next PCM bytes, a whole number of frames. | | sentence | {"index", "text", "seconds"} after the last audio of each sentence. | | done | {"take id", "frames", "seconds"} once the take is complete. | | error | {"message"} , after which the stream ends. | The route still needs the bearer token, and the server sends no CORS headers, so a browser's EventSource , which cannot send an Authorization header, does not work with the token on. Read the events with fetch and a stream reader from a page the server already trusts, or with any HTTP client. curl -sSN -H "$AUTH" "localhost:8780/v1/audio/speech/stream?input=Hello%20there.%20Good%20to%20see%20you." Live text input. POST /v1/audio/speech/live is for an agent that writes text a piece at a time. It takes voice , expressiveness , seed and the character fields as query parameters, and the text as the request body: - Send the body with Transfer-Encoding: chunked , or with a Content-Length that covers everything you will send. The server reads it as it arrives. - The body is UTF-8 text in lines ended by \n or \r\n . The server acts on a line once its newline arrives. - The server joins the lines with spaces and speaks each sentence once a sentence end . , , ? , ; , … , or a CJK one, optionally followed by a closing quote or bracket arrives. Sentences are cut with the same splitter as input , so <|speaker:N| tags and cues work. - The pronunciation rules rewrite each sentence before it is spoken, unless the query has apply pronunciation=false . - An empty line speaks the text held back so far, finished or not. The end of the body does the same, and a client may half-close its side of the connection to end the body. - Each sentence is a take, and the session's takes carry the voice and the history: the first take is the voice of every later sentence when no voice is given, and the last two are context. - The response is chunked PCM as for stream: true . Send Accept: text/event-stream to get the events above instead, with done carrying {"sentences", "seconds", "take ids"} . The response ends once the body has ended and its last sentence is spoken. - The session holds the engine from the request to the end of the response, so other requests get 429 meanwhile. tools/live speech.py sends each line of its standard input as one chunk as soon as it reads it, and writes the PCM to standard output: some-agent | tools/live speech.py --voice Narrator | ffplay -f s16le -ar 44100 -ch layout mono -nodisp - tools/live speech.py --events < script.txt prints each event with its arrival time curl -T - also speaks a piped body, but it holds the piped lines back before sending them: on the mock engine its first audio came 1.7 s later than the same feed through tools/live speech.py . Manage voices: curl -sS -H "$AUTH" localhost:8780/v1/voices curl -sS -H "$AUTH" localhost:8780/v1/voices -F file=@clip.wav -F name=Narrator -F "transcript=What the clip says." curl -sS -H "$AUTH" -X DELETE localhost:8780/v1/voices/<id tools/api smoke.sh base-url runs all of the above against a running server and checks each answer. drama-mcp is an MCP server over stdio protocol 2025-06-18 that calls the HTTP API. tools/make app.sh puts it at DramaStudio.app/Contents/MacOS/drama-mcp ; swift build puts it at .build/debug/drama-mcp . When nothing answers, it opens Drama Studio and waits up to 90 seconds for /v1/health . It looks for the app in DRAMA APP PATH , then the app that contains it, then /Applications/DramaStudio.app , then the repository's dist/DramaStudio.app . DRAMA API URL overrides http://127.0.0.1:8780 . It reads the token from the same file. | Tool | Does | |---|---| | synthesize speech | text , with optional voice or character {name, description, scene?} , output path , expressiveness , and play . Writes a WAV, by default to ~/Library/Application Support/DramaStudio/Output/<timestamp .wav . Returns the path, seconds, frames, generation seconds, and take id . play plays it with afplay . | | retake with note | take id and note , with optional output path and play . Generates that take's line again in the same voice with the note. Returns what synthesize speech returns, with the new take id . | | list voices | The voice library. | | add voice | Clones audio path with its transcript under name . | | delete voice | Deletes a voice by id . | | studio status | Health, the model snapshot, and the API address. | There is no MCP tool for streaming or live text input. An MCP tool call is one request and one result, so it cannot carry audio while text is still being written; an agent that wants that calls POST /v1/audio/speech/live itself. Register it with Claude Code: claude mcp add drama-studio -- /Applications/DramaStudio.app/Contents/MacOS/drama-mcp Or with Claude Desktop, in ~/Library/Application Support/Claude/claude desktop config.json : { "mcpServers": { "drama-studio": { "command": "/Applications/DramaStudio.app/Contents/MacOS/drama-mcp" } } } drama-say speaks through the running app the way /usr/bin/say speaks through macOS. It sends POST /v1/audio/speech , finds and opens the app the way drama-mcp does, and loads no model of its own. tools/make app.sh puts it at DramaStudio.app/Contents/MacOS/drama-say . drama-say "Hello there. laughs Good to see you." drama-say -v Narrator -o hello.wav -f chapter.txt drama-say --character "Captain: gravelly old male voice" --verbose "Pray ye never find out." drama-say --retake "$TAKE" --note "softer and warmer" With no -o , it plays the line as it generates: it sends stream: true and plays each completed chunk when it arrives. Startup time depends on model loading, the script, the voice and retries; it is not a fixed delay. -o FILE writes a WAV, M4A or PCM file, chosen by the extension, and -o - writes raw PCM to standard output. With no text and no -f , it reads the text from standard input. -v '?' lists the voice library. --verbose prints the take id that --retake takes. drama-say --help lists every option. It prints nothing on success. It exits with 64 on a bad command line, 75 when the studio is busy with another generation, after printing the Retry-After wait, and 1 on any other error, after printing the server's message. DramaEngine is a Swift library, version 2.1.2 DramaEngineVersion.string , CHANGELOG.md . Add this repository as a package dependency and the DramaEngine product to a target; the app, the server and the CLI are not needed. The public API includes ModelLocation and DramaSetup for the install, DramaSynthesizer an actor behind the SpeechSynthesizer protocol, SpeechRequest with ReferenceVoice , VoiceDesign , Take and GenerationConfig , the streamed SpeechEvent s with StreamedAudio and GenerationEstimate , PronunciationDictionary , DramaCodec for standalone encode and decode, and MockSynthesizer for previews and tests. Everything else in Sources/DramaEngine is package : the repository's targets use it, other packages cannot. python import DramaEngine let report = DramaSetup.check location: .huggingFaceCache guard let location = report.location, report.isReady else { fatalError report.fixes.joined separator: "\n" } let engine = DramaSynthesizer location: location, codecDirectory: DramaCodec.defaultDirectory let voice = try await engine.encodeReference audioURL: clip, transcript: "What the clip says.", name: "Narrator" var request = SpeechRequest text: "Hello there.", references: voice request.config.streaming = .defaultLowLatency request.config.maxRetries = 0 // Allow partial playback before a chunk is validated. var audio = StreamedAudio for try await event in engine.synthesize request { play audio.newSamples from: event } - DramaSetup.check reads files only and names what is missing: a weight shard, the tokenizer, codec.pth , the Core ML codec models, or a tuned kernel schedule for this Mac, each with a fix. DramaSetup.prepare builds the codec models with the convert codec.py that ships in the engine's resource bundle, through a Python you name it throws the exact commands when you name none and never downloads Python , and loads the model once to tune the kernels. - Examples/SpeakCLI is a separate package that depends on this one by path, clones a voice from a clip, streams progress, and writes a WAV in 67 lines. Examples/README.md has the Xcode steps and the entitlements: none beyond the app's own, plus the microphone only to record clips. - The DocC catalog Sources/DramaEngine/DramaEngine.docc has the landing page and the articles "Getting the model", "Voices" and "Streaming". Build it with xcodebuild docbuild -scheme DramaEngine -destination 'generic/platform=macOS' -derivedDataPath .build/docbuild ; no DocC plugin is needed. swift test --filter SDKUsageTests runs the landing page's example on MockSynthesizer and fails if the page's copy drifts. - tools/api audit.py after swift package dump-symbol-graph --minimum-access-level public lists the public symbols and fails on one without a doc comment or a public value type that is not Sendable . - tools/make xcframework.sh builds dist/DramaEngine.xcframework for macOS on Apple silicon, with the Metal kernel sources and the conversion script inside the framework, then builds and runs SpeakCLI against it. - Sources/DramaEngine/Weights : the eight safetensors shards are memory-mapped MAP SHARED and wrapped as Metal buffers without copying. Experts stay in FP8 E4M3 with their 128x128 block scales and are decoded inside the kernel through a 256-entry table; every other weight is bf16 widened to fp32 in the kernel. - Sources/DramaEngine/Kernels : zero-centered RMSNorm, a bf16 GEMV for decode and a simdgroup-matrix GEMM for prefill, the Gated DeltaNet depthwise conv and recurrence, GQA attention with an online softmax over an fp32 KV cache, partial RoPE NeoX for the backbone, interleaved for the tower , the MoE router and expert-major FP8 kernels that dispatch only over active experts one-row kernels for decode, tiled simdgroup-matrix kernels for prefill , the GPU sampler, and the codebook tower's embedding mixing. Tile shapes, threadgroup sizes and the attention key split are fields of a KernelSchedule that Autotune fills per machine see below . - Sources/DramaEngine/Model : the backbone and tower runners, the reference's sampler top-k, top-p, exponential-noise argmax, repetition-aware retry of the main token on the CPU and as Metal kernels that reproduce its picks and generator stream bit for bit, prefill and decode sessions, and the chunked frame generator. A decode frame is one command buffer, with the next frame's buffer committed before the current one is waited for. - Sources/DramaEngine/Prompt : the reference-bracket-v1 prompt builder, the reference's chunker and history policy, plus a sentence-per-turn mode. - Sources/DramaEngine/Tokenizer : a byte-level BPE tokenizer that matches Hugging Face tokenizers on a 661-case corpus. - Sources/DramaEngine/Codec : Core ML wrappers for the DAC encoder reference clips to codes and decoder codes to 44.1 kHz audio , both exported at one static 512-frame shape and run in windows with 256 frames of left context, plus the same decoder at a 64-frame shape for the silence stop's probe. - Sources/DramaServer : the HTTP API on Network.framework . Request parsing and routing are pure functions from an HTTPRequest to an APIRequest ; APIService answers them against any SpeechSynthesizer and VoiceStore , so the app, drama serve , and the tests share it. - Sources/DramaMCP and Sources/drama-mcp : the MCP server, JSON-RPC 2.0 over stdio, proxying the HTTP API. - App/DramaStudio : the SwiftUI app, with the API server and the menu bar extra in App/DramaStudio/Server . - eval/ : the faithfulness suite see docs/EVAL.md . Current UI checks and responsiveness measurements are in UI verification https://github.com/maralski/drama-studio/blob/main/docs/ui-verification.md . Historical engine experiments below record their original configurations; they are not current latency guarantees. Design notes and decisions are in docs/DESIGN.md ; the decision trail of the build is .audit/drama-studio.tsv . The engine is checked against the fish-speech v3.0.0-beta PyTorch implementation running on the CPU in float32, teacher-forced on the same prompt and frames. Numbers from eval/compare.py : | quantity | relative L2 vs float32 oracle | top-1 agreement | mean KL | |---|---|---|---| | backbone hidden state, 121 positions worst position 1.3e-6 | 5.7e-7 | | | | main-token candidate logits | 3.0e-7 | 1.000 | 1.3e-12 | | nine tower head logits | 4.4e-7 | 1.000 | 4.2e-12 | | residual stream after each of the 40 layers, and each of the 13 tower layers, at the first step | under 2e-6 each | | | For scale, the same oracle run in bfloat16 on the CPU differs from its float32 run by a relative L2 of 5.2e-2 on the hidden state with a top-1 agreement of 0.846. The Metal engine sits at float32 precision; it keeps the FP8 expert weights exact and widens everything else, so it does not reproduce the CUDA release's W8A8 activation quantization, and no claim is made about that. The sampler is checked separately: over 1174 sampled picks, none falls outside the reference's top-k / top-p set, and the count of rank-0 picks matches the expected value within noise. End-to-end, the sentences the model does speak transcribe cleanly under Whisper, but with the reference's chunking it often stops after the first sentence of a two-sentence prompt or trails into silence, and the reference PyTorch implementation does the same on the same text and seeds. The sentence-per-turn mode and the retry rule below exist for that reason. See eval/README.md for how to reproduce every number. The codec uses Core ML to access Apple's inference backends. In the recorded experiments on macOS 26.5 and an M3 Max, the Neural Engine was slower than the CPU: - The decoder's Neural Engine compile for the 512-frame shape ran for more than 35 minutes without finishing; the 64-frame shape compiled in about 7 minutes and then ran three times slower on the ANE than on the CPU, because the compute plan puts 72 percent of the decoder's cost long-sequence convolutions and the Snake activation's sin after upsampling on the CPU anyway. - The ANE's native transposed convolution lost about 15 dB of accuracy, so the export rewrites transposed convolutions as polyphase convolutions. - The GPU path returns wrong audio for the chained residual units of the last decoder block about 5 dB SNR ; each unit alone is correct, which points at op fusion, though the cause was not isolated. - Core ML's enumerated shapes crash the CPU backend SIGTRAP on multi-channel convolutions here, so the models use one static shape. So the shipped configuration runs the encoder in fp32 and the decoder in fp16 on the CPU through Core ML: 1.5 s per 512-frame window 23.8 s of audio for the decoder and 2.1 s for a 10 s reference clip in the encoder. The encoder matches PyTorch's codes exactly; the decoder is 35 to 39 dB from the fp32 reference. DramaCodec.init computeUnits: still accepts .cpuAndNeuralEngine for anyone who wants to retry on a different machine or OS. Measured with eval/e2e.py on six texts, plain and cloned, three seeds each 36 runs per row , scored by Whisper small.en word error rate, whether the model reached eos, trailing silent frames, and resemblyzer speaker similarity to the reference clip for the cloned runs. | configuration | mean WER | runs at or under 0.1 WER | eos reached | mean quiet tail | speaker similarity | |---|---|---|---|---|---| | reference chunking 300 bytes , no probe | 0.141 | 69% | 89% | 103 frames | 0.867 | | one sentence per turn | 0.063 | 86% | 100% | 30 frames | 0.867 | | sentence per turn, silence stop, short-chunk retry previous defaults | 0.016 | 92% | 100% | 2 frames | 0.867 | The app uses the shipped chunking and retry policies. The drama CLI exposes --chunking for unchecked comparisons; checked generation always uses sentence turns. --no-retry and --max-frames remain available. Uncloned outputs score 0.54 to 0.56 against the reference clip, so the cloned 0.87 is the voice carrying over, not the metric's floor. The only misses in the last row are Whisper rendering "the 21st of March" as "21 March" in three runs; every other run transcribes exactly. The retry rule's threshold was chosen on the same 72 runs of the first two rows and then applied to the third, so treat it as tuned to this sentence set. The word error rates above use the PyTorch codec to decode. Decoding the same 36 frame sequences with the shipped Core ML fp16 decoder instead eval/coreml vs pytorch wer.py gives the same mean word error rate, 0.063 against 0.063, with no run scoring worse. Medians of five interleaved runs against the previous engine on an idle machine eval/perf ab.sh , raw runs under eval/out/perf/ab final/ . GPU times come from drama bench ; the end-to-end rows are wall time from drama generate --warm on a 324-row prompt a 10 s reference clip plus text and 600 frames. | step | before | now | |---|---|---| | weights mapped and kernels compiled | 0.4 to 0.7 s | 0.4 to 0.7 s | | warm-up page walk, pipelines, first command buffer | 4 s cold, then a 0.37 s first forward | 4 s cold; the first forward then costs 0.05 s | | backbone prefill, 128 rows | 647 ms GPU | 213 ms GPU | | tower prefill, 128 rows | 171 ms GPU | 38 ms GPU | | backbone plus tower prefill, 300 rows in one chunk | three 128-row chunks | 382 plus 93 ms GPU | | end-to-end prefill of a 324-row prompt | 2.44 s | 0.72 s | | backbone decode, one frame | 17 to 18 ms GPU | 17 to 18 ms GPU | | tower decode, one frame | 6.7 ms GPU | 6.7 ms GPU | | decode, one frame | ten command buffers, sampling on the CPU between tower layers | one command buffer, 25.6 to 27.0 ms GPU, the next frame's buffer committed before the current one is waited for | | end-to-end decode | 31.5 frames/s range 28.7 to 31.6 | 38.5 frames/s range 37.0 to 39.1 , 1.8x real time | Prefill runs dense projections through a simdgroup-matrix GEMM and the FP8 experts through tiled kernels that dequantize each weight tile once; at 128 rows nearly every expert is active, so a chunk reads all 30 GB of expert weights once and the chunk is 512 rows so a 10 s reference clip is one pass. Small chunks pay for that choice: an 8-row chunk takes 66 ms instead of 42, because one tile shape serves every row count so that a row's result is the same in any chunk the prefix cache depends on it . Decode is bound by weight bandwidth: the one-row backbone forward reads about 3.4 GB of weights in 17 to 18 ms, about 200 GB/s averaged over its 600 small dispatches, and the no-experts bench delta puts the eight active experts at about 250 GB/s. drama profile --rows N prints the GPU time of each kernel group on the real weights one layer repeated, so its numbers can be cache-warm , and drama kbench times the GEMM tile shapes and the expert kernels on synthetic data. The before column comes from the same session as the now column; the same baseline binary measured 583 to 586 ms for the 128-row prefill in two earlier series that day, so day-to-day drift of about 10 percent is normal. The kernel changes keep every fp32 product and change only summation orders; the faithfulness table above is from the new engine hidden 5.7e-7, candidate logits 3.0e-7, tower logits 4.4e-7 relative L2, top-1 1.0; it was 6.5e-7, 3.3e-7 and 5.0e-7 before . The GPU sampler reproduces the CPU sampler's picks and generator stream: 160,000 chained picks identical in the long parity run on the pre-merge code DRAMA SAMPLER PARITY SAMPLES=20000 , 16,000 in the default run on the merged code, and every generation compared between the CPU and GPU samplers on the previous kernels produced the same frames. Reordered sums do change the result on some prompts: in the end-to-end suite 28 of 36 runs produced the same frames as before, one differed only in where the codec silence probe stopped it, and 7 differed deterministically, with the summary metrics unchanged mean word error rate 0.016 in both, eos reached in all runs, speaker similarity 0.870 against 0.867 . On the one such prompt examined eval/specs/qshort.json , the two engines agree to 2e-7 through layer 29 and part at layer 35, the shape of a discrete routing change on a near-tie the router ids themselves were not dumped . A fresh float32 oracle dump of that prompt takes the new engine's side: 9.5e-7 on the hidden state, where the previous engine was off by 3.9e-3 and failed the suite's thresholds. That holds for this one prompt only; the faithfulness gate above runs on a single prompt and would not catch the same kind of divergence on another. A second round on the decode frame, run as a hillclimb: seven experiments in small synthetic settings first, then one change at a time into the real frame, each kept only when the output stayed bit-identical and the frame got measurably shorter. The decision log is .audit/hillclimb.tsv . What the experiments established before anything changed. A decode frame was 1312 dispatches streaming 5.94 GB of weights in 21.7 ms, and its large GEMVs already ran at 346 to 378 GB/s against a measured read ceiling of 383 GB/s. At that rate the bytes alone take 15.5 ms, so the 30 percent target of this round 14.6 ms sits below what any kernel change can reach; fewer bytes per frame were the only route, and the only route that keeps the output exact is lossless compression. The bf16 mantissas carry 6.97 of 7 bits and are incompressible; the exponents carry about 2.8 bits of 8, so a 3-bit exponent code with a per-tile window and byte escapes packs a weight into 11.3 bits with exact round trip. The kernel that unpacks it is compute-bound at about 285 GB/s of physical reads, so it wins only on the K=2048 shapes with wide N. What shipped, every step with a byte-identical dump and the faithfulness table unchanged hidden 1.2e-6 relative L2 at most, top-1 1.0 : | step | change | frame bench rows 1, backbone + tower , under load | quiet GPU, 2026-10-23 | |---|---|---|---| | baseline | | 21.1 ms | 21.0 ms 15.0 + 6.0 | | MoE layer | one decode row's mixture of experts as three dispatches bf16 router and shared pass, routing plus routed gate-up, routed down plus shared down plus combine , FP8 decoded with integer ops instead of the lookup table | 20.2 | 20.1 | | encoder | a concurrent compute encoder with a barrier before every dispatch except the later members of a sibling group the four GDN input projections, q k v, w1 and w3, and so on ; q/k norm, rope and the KV writes in one dispatch | 19.7 | 19.5 | | samplers | 256-thread sampling kernels in place of one 1024-thread threadgroup; no change on a quiet GPU, and 47 to 22 ms per frame when another GPU client is active, which is what a user with an animating window sees; on a quiet GPU the kernels cost 0.67 ms per frame, about 0.2 ms more than the old shape, and the old shape stalls under the window server alone | 19.7 | not re-benched | | tower | embedding plus both concat norms in one dispatch, attention split over 8 threadgroups with an in-order combine and the gate folded in, the residual adds folded into wo and w2, w1 and w3 as one dispatch with the SiLU | 19.6 | not re-benched | | packed weights | eight weight families GDN qkv and z, attention q, the semantic lm-head slice, tower wqkv, heads, w1 and w3 read from an exponent-coded copy at one row, 0.94 GB fewer bytes per frame | 18.2 | 18.1 12.9 + 5.2 | Reverted: a multi-matrix GEMV that grouped sibling projections into one dispatch within noise once the concurrent encoder had taken the same gaps , more threadgroups for the MoE down kernel each doubling cost 3 to 9 us per layer in its prologue , and the load folds of norms, gates and SiLU into the GEMVs no gain on the tuned schedule . The frame went from 1312 dispatches to 759. The packed copy lives under ~/Library/Application Support/DramaStudio/Packed/ , 2.1 GB per checkpoint, built on the first load in about two seconds across all cores and verified by a full unpack before it is used; a cache that fails to build or load falls back to bf16 with the reason logged, and DRAMA PACKED=0 turns it off without a rebuild. Prefill, chunk invariance and the prefix cache never touch it: every path with more than one row reads the bf16 shards as before. Caveats, and what a quiet session settled 2026-10-23 . Every number in the hillclimb was taken while another process held the GPU the window server and a virtual machine , so the ranges carried stalls and no end-to-end figure was quoted. A second session rechecked the round with the virtual machine stopped and the app quit, one job at a time. The ioreg utilization counter reads 100 percent under the window server alone and is no gate; the gate was the sampler stages of a traced 120-frame generation eval/quiet label.sh , 0.7 ms per frame when the GPU is free, taken before and inside every batch. The tune was redone there: the decode GEMV moved to 8 simdgroups, the narrow GEMV kept its tile this time, the MoE prefill tile went to 32x32x32, and the 128-row forward went from 233 to 189 ms. The quiet column above is a 10-round interleaved bench per pair except the baseline, 5 rounds. The concurrent encoder stays: against a serial-encoder build of the same code it is 0.5 ms shorter on the bench line and 0.0 to 1.2 ms shorter per frame over three interleaved 300-frame traces, with a fatter tail p99 39 to 41 ms against 33 to 35 , which playback's 64-frame buffer absorbs. The parked multi-matrix GEMV was rebased branch hill/step3-gemv-decode-quiet and measured again: 0.1 ms, inside the A/A gap of the bench, so it stays parked. End to end, eval/perf ab.sh over five 425-frame generations: 35.1 frames per second 31.7..36.4 against 20.7 for the baseline, whose 1024-thread samplers stall under the window server alone. Short runs after idle reach 48 to 50 frames per second 19.2 to 19.7 ms per frame ; the sustained figure is lower because this Mac slows bandwidth-bound work by 20 to 40 percent after minutes of GPU load and recovers over minutes of idle, so absolute decode numbers depend on the GPU's recent history while interleaved A/B deltas hold. The remaining ideas are small: the MoE down kernel's prologue, and the w2 and 4096x2048 families in the packed path. Default sentence chunking keeps short complete sentences separate. It no longer folds a short opening sentence into the next one, which could let the model end after the opening and omit the rest of that chunk. GenerationConfig.streaming requests when decoded audio leaves the engine. When maxRetries and minSpokenRatio are positive, every request uses .sentence mode. Each chunk is decoded and checked before its selected attempt leaves the engine. Checked generation also verifies both complete neighboring chunks together before exposing their join. A mismatch retries only the current unplayed chunk within the existing attempt limit. Already emitted audio stays fixed. This applies to every voice, plain text, and performance cues. With explicitly unchecked preview and retries disabled, .lowLatency stepFrames: 64 sends partial audio every 64 generated frames, about 3 seconds. The CPU decodes those frames while the GPU generates the next ones. Such a chunk cannot be replaced after playback begins. Normal generation waits for the model's end-of-speech signal, subject to its frame cap and loop guard. It does not treat silence as completion, because a voice can pause before the remaining words. Explicit silence probes run at fixed frame checkpoints and finish before sampling resumes, so decode timing cannot change the stop frame. Normal generation checks the decoded words with on-device speech recognition. Missing, changed, or uncertain words trigger a retry before audio enters playback or generation history. Cue retries alternate the requested sampling with broader replacements and ask for precise enunciation. Exhausted retries report a failure. Reference pace affects the time allowance, not the words that must be spoken. Explicit recognition labels such as gasp or gasps count as nonverbal annotations only when the script requests that cue family. Ordinary spoken words remain required. The native checker selects an installed regional locale from your macOS language preferences, with a deterministic fallback. Acceptance policy revision 5 records the verified-sentence boundary rule. Archived takes retain their original check revision; playback does not upgrade their verification history. Opt-in local audits record the exact PCM identities, unit indices, token and confidence-span positions used by the boundary check. The decode reuses the windows a full decode of the chunk would run DecodeWindow.covering , so a chunk's partials join into exactly its audio. On the Core ML codec, 600 frames decoded in 64-frame steps differ from one full decode by a maximum absolute error of 0 DRAMA CODEC SEAM=1 swift test --filter testSeamOnTheRealCodec ; decoding each step with a bare 256-frame left context instead differs by 0.027. All full-window codec work of a run goes through one serial queue: a low-latency chunk's last frames decode while the next chunk prefills and generates. The optional silence stop reads fixed frame checkpoints through the 64-frame probe shape. Each reading completes before generation samples another frame. Readings start after the voice-aware expected duration plus the configured silent tail. drama generate --probe-every N and --probe-after-expected no explicitly enable the probe for measurement. In sentence mode, each attempt's full audio is decoded once at its end. The measurements below describe the earlier asynchronous streaming implementation. They predate universal retry buffering and the removal of silence stopping from normal generation. eval/reliability/REPORT.md records the earlier probe timing experiments. Measured on the M3 Max 16 cores, 12 performance , with the same two-sentence line, 9.4 s of audio in 202 frames, seed 7, in every run. The machine was not idle: other users' jobs kept the load average at 5 to 7 two Python processes at a full core each , so every comparison interleaves its sides, and the raw rows are in .audit/streaming.tsv . Time to the first audio byte of stream: true through curl -N tools/streaming ttfb.py , 5 runs per side, alternating . Before is the engine at 9e645ef , which decoded every chunk after the whole request was generated. Both sides returned the same 827,392 bytes, with HTTP 200 and curl exit 0 every time. Server CPU comes from ps -o time= before and after each request: | | before | after low latency | |---|---|---| | first audio byte | 12.95 s 12.61 to 13.20 | 4.49 s 4.48 to 4.60 | | last byte | 14.48 s 14.14 to 14.72 | 8.57 s 8.49 to 8.68 | | server CPU per request | 39.0 s | 22.6 s | | average cores in use | 2.69 | 2.65 | | peak cores over any 0.5 s | 4.3 3.8 to 4.7 | 4.0 3.8 to 4.5 | The first byte now comes after the prefill 0.7 s , 64 frames 1.8 s , and one decode of a 512-frame window about 2 s . The request also uses less CPU: before, the silence probe decoded the whole chunk so far every 16 frames whenever no probe was running, and every chunk was decoded once more at the end; now each window is decoded once. Generation speed with the decode on and off, in one process DRAMA STREAMING BENCH=1 swift test -c release -Xswiftc -enable-testing --filter StreamingBenchTests , 5 runs per mode, modes interleaved, medians with ranges . "Decode after" turns the silence stop off, so the codec is idle while frames land and runs once per chunk at its end: | mode | frames/s while frames land | end-to-end frames/s | first audio | wall | CPU cores | |---|---|---|---|---|---| | decode after | 34.9 34.7 to 37.5 | 19.1 | 5.07 s | 10.58 s | 1.20 | | sentence | 33.0 32.2 to 33.7 | 15.2 | 6.51 s | 13.30 s | 2.03 | | low latency, 64 frames | 32.7 32.4 to 33.6 | 18.9 | 6.77 s | 10.68 s | 2.11 | | low latency, 32 frames | 32.9 32.8 to 33.5 | 19.2 | 6.03 s | 10.50 s | 2.17 | - The CPU decode slows generation by about 6 percent while it overlaps the GPU 34.9 to 32.7 frames/s . Low latency still finishes with "decode after" and 2.6 s ahead of sentence mode, because the GPU never waits for the codec. - What limits the first audio is the decode, not the GPU. One window takes 1.6 s alone "decode after" spends 3.35 s on two and 4.6 s while the GPU generates: in the event trace of the first low-latency run, the window was queued at 2.50 s and its audio arrived at 7.09 s. The cause of the slowdown was not isolated; contention for memory bandwidth or for the chip's power budget is a guess. The test runner also sees slower decodes than drama serve first audio 6.77 s here against 4.49 s through the server , and that cause was not isolated either. - For this short first sentence 4.7 s of audio , low latency is no faster than sentence mode to the first audio. It pays off on a long first sentence: on one 12.5 s sentence 3 runs , low latency started at 6.19 s 6.09 to 6.94 against 23.5 s for sentence mode. Part of that gap is the retry: sentence mode judged the first attempt too short and generated the sentence again, while low latency kept that attempt 336 to 369 frames . - 32-frame steps start 0.7 s sooner but stutter: in every trace the first 1.5 s of audio ran out 1.1 to 1.5 s before the next partial arrived. With 64-frame steps every trace played without a gap, so 64 stays the default. The app shows the time left beside the elapsed time while it generates GenerationEstimate : the expected frames of the unfinished chunks text length at 16 characters per second, or the reference voice's slower pace, times 21.53 frames per second less the frames of the current chunk, over a 3 s moving average of frames per second that starts at 35, plus one prefill per chunk not yet prefilled 0.7 s until the first is measured . It reads "about 12 s left", rounded to 5 s above 20 s, and "finishing…" or "retrying" once the frames stop predicting the end. The same numbers are X-Drama-Estimated-Seconds and the estimate field of /v1/health . Kernel schedule parameters the GEMV token tile and rows per threadgroup, the GEMM tile, the MoE tile and K step, the one-row expert kernels' rows per threadgroup, the attention key split, the GDN threads per state column are not constants any more. On the first load for a given chip, macOS version, kernel source digest, model snapshot the engine times each kernel family's real workload on the resident weights, rotated over ten layers, for every value in its declared domain, keeps a candidate only when its output is within a relative L2 of 1e-5 of the default's on the same inputs and it is at least 5 percent faster over five interleaved rounds, then times whole forwards of 128 and 300 rows and of one row on both schedules and drops a side's choices unless the tuned one wins by 2 percent at every row count. The result is a JSON report under ~/Library/Application Support/DramaStudio/Tuning/ ; every later load reads it and measures nothing. drama tune runs it from the command line and prints the table about 30 s here ; the app shows "Tuning kernels for this Mac" once; drama schedule-ab times any two schedule files against each other inside one process. The idea is Magnitude's tune, validate, cache per device ; the kernels, the domains and the measurement scripts are this project's. Nothing in the numerics changes beyond summation order: the tuned engine scores hidden 5.8e-7, candidate logits 3.0e-7 and tower logits 4.4e-7 against the float32 oracle with top-1 1.0 eval/out/perf/compare tuned final.txt , and the A/B prompt below produced byte-identical frames with and without the tuned schedule in all fourteen compared runs. What it shipped on this M3 Max eval/out/perf/tune final2.txt : two decode changes, a token tile of 1 with two simdgroups per threadgroup for the one-row GEMVs 27 percent faster per kernel and a 16-way key split for one-row attention 27 percent . Everything on the prefill side stayed at the hand-picked values: the GEMM tile, the MoE tile and the GDN split never cleared the 5 percent bar, and the narrow-projection GEMV tile and the 4-way prefill attention split, 80 and 5 percent faster per kernel, moved the whole 128- and 300-row forwards by 1.2 and 0.4 percent, under the margin, so the whole-forward check reverted them. Two earlier versions of the measurement script would have shipped a slower prefill: random activations route evenly over the 256 experts and favour a small MoE row tile that was 9 percent faster in the measurement script and 5 to 12 percent slower in the real prefill, and a 2 percent minimum gain measured at 128 and 32 rows let through a tile that lost 9 percent at 300 rows. Both are in .audit/autotune.tsv , and the isolation that caught the second is eval/out/perf/schedule ab 1.txt . Measured, with the shipped schedule against the default: | measurement | untuned | tuned | |---|---|---| | backbone forward, one row, GPU, drama schedule-ab in one process, 7 interleaved rounds | 19.2 ms 18.4..19.5 | 15.3 ms 14.8..15.5 , faster in 7 of 7 | | same, drama bench in separate processes, two five-run series | 18.0 17.2..18.3 , 17.9 17.5..18.2 | 15.1 14.7..15.8 , 15.4 14.8..16.7 | | tower decode, one row, GPU, drama bench | 6.9, 6.8 ms | 6.1, 6.2 ms | | end-to-end decode, 425 frames after a 324-row prompt, drama generate , five-run medians | 32.9 15.0..37.1 , 29.7 21.8..33.1 frames/s | 33.5 14.9..42.2 , 35.1 32.3..36.8 frames/s | | same, GPU ms per frame, the three rounds of the first series where neither side was disturbed, then all five of the second | 26.9, 27.9, 30.4; 46.0, 34.6, 32.9, 33.7, 30.2 | 24.6, 23.7, 29.8; 31.0, 28.5, 27.2, 27.7, 29.7 | | backbone prefill, 128 and 300 rows | unchanged: the shipped schedule runs the same kernels schedule-ab 196.9 against 201.0 and 382.9 against 382.0 ms, the instrument's own spread | | The one-row forward is the solid number: about 20 percent less GPU time in four series the two bench series, the in-process A/B and the tuner's own check at 17.1 against 14.6 ms , ranges not overlapping, from the decode GEMV tile and the attention split, both of which put more and smaller threadgroups on the same bytes. The end-to-end frames-per-second rows are softer: the two drama generate series ran on a machine that was not idle other sessions' test jobs and the window server busy; load average 4.8 to 12 , their ranges are wide, and in the first series two of five rounds were disturbed on both sides for a reason the load averages do not show, so the five-run medians move only 2 and 18 percent while the per-frame GPU time is lower on the tuned side in every round where neither side was disturbed. Those two series ran with an earlier cache whose decode fields equal the shipped ones and whose prefill fields have since been reverted. Prefill has no measured change and none is expected: with the reverted fields the schedule is the default there. To redo the end-to-end table on an idle machine: eval/tune ab.sh .build/release/drama 5 eval/out/perf/ab tune idle then eval/.venv/bin/python -I eval/perf ab summary.py eval/out/perf/ab tune idle . Copyright 2026 Peter Marelas. Drama Studio's own code is under the Apache License 2.0; see LICENSE https://github.com/maralski/drama-studio/blob/main/LICENSE and NOTICE https://github.com/maralski/drama-studio/blob/main/NOTICE . Built with Fish Audio. Drama Studio runs Fish Audio Drama 3, and the model weights and codec checkpoint are not included: the app downloads them from fishaudio/drama-3 https://huggingface.co/fishaudio/drama-3 , where Fish Audio licenses them under the Fish Audio Community License, Version 1.0, copied here as LICENSE-FISH-AUDIO https://github.com/maralski/drama-studio/blob/main/LICENSE-FISH-AUDIO . That license governs the model and the audio you make with it. It allows research and non-commercial use, and commercial use while annual revenue is under USD 500,000; it forbids cloning a real person's voice without their consent and the other uses its Section IV b lists, and it needs a separate Fish Audio license to offer speech generation to others as a hosted service. Section IV b of that license asks anyone who makes speech generation available to others to bind them to its use restrictions, so the app's setup window shows them and asks you to agree before it downloads or generates anything. Until you agree, the window, the HTTP API 403 and through it the MCP server and drama-say refuse to speak. The command-line drama tool is for developers building from source and does not ask. The fish-speech reference code copied under eval/oracle/ and Sources/DramaEngine/CodecConversion/dac reference/ , the parts of DramaEngine reimplemented from it, the converted Core ML codec, and the example cast from Fish Audio's Drama 3 Space stay under the Fish Audio Community License as well; THIRD PARTY NOTICES https://github.com/maralski/drama-studio/blob/main/THIRD PARTY NOTICES lists each one and what changed. descript-audio-codec code keeps its MIT notice there. This project is not affiliated with or endorsed by Fish Audio. During setup, you can choose to share anonymous usage information to help improve Drama Studio. Sharing is off unless you enable it. Only counts of app launches, generation outcomes and setup failures are included. Scripts, audio, voices, file names and personal identifiers are excluded. Change your choice at any time in Settings General Privacy. See usage monitoring configuration https://github.com/maralski/drama-studio/blob/main/docs/usage-monitoring.md for release setup and the data contract.