{"slug": "drama-3-shipped-one-hour-later-we-ran-it-63-faster-than-day-zero-support", "title": "Drama 3 shipped. One hour later, we ran it 63% faster than day-zero support.", "summary": "INT21 reported that its self-improving agent swarms produced a working Rust inference runtime for Fish Audio's Drama 3 speech model one hour after the model's release, delivering 1.59–1.63× the throughput and 38–60% lower time to first audio than SGLang-Omni, which had day-zero support. On 2× B200 GPUs, the INT21 runtime reached 15.11–23.58 ms time to first audio and 223.4–249.2 frames per second across three prompts, versus 37.79–38.61 ms and 136.9–154.9 frames per second for SGLang-Omni. Drama 3 combines FP8 mixture-of-experts, codebook multi-token prediction and linear attention, three techniques INT21 said standard speech runtimes are not built for.", "body_md": "## Key highlights\n\n- \n**One hour** from model access to a working Rust inference runtime, with no early access.\n- \n**1.59–1.63× the throughput** of SGLang-Omni, which had day-zero support.\n- \n**38–60% lower time to first audio** , with first speech in 15–24 milliseconds.\n\n## What happened\n\nFish Audio released [Drama 3](https://huggingface.co/fishaudio/drama-3) with [day-zero SGLang-Omni support](https://sgl-project.github.io/sglang-omni/cookbook/fishaudio_drama3.html).\n\nOur [Inference Engine Factory](https://drama3-newwebsite.bing-xu-mail-e57.workers.dev/products/inference-engine-factory/) has generated more than 20 engines to date. Drama 3 combines three techniques that standard speech runtimes aren’t built for: FP8 mixture-of-experts, codebook multi-token prediction and linear attention. Each needs custom engineering to run fast. Tackling a new architecture on release day made Drama 3 one of our hardest tests yet.\n\nYet only one hour after we started, our self-improving agent swarms had a working Rust runtime that outperformed SGLang-Omni on every prompt tested.\n\n## Benchmark results\n\nBoth runtimes used 2× B200: one GPU for generation, one for the codec. Each test used one speech stream, a 1-frame first chunk, and 20 warm trials per prompt.\n\nWe measured time to first audio, the delay before the first chunk of speech arrives, and throughput, how many audio frames the runtime generates per second. Lower latency and higher throughput are better.\n\n| Benchmark | Runtime | Time to first audio ↓ Lower is better | Throughput ↑ Higher is better | RTF ↓ Lower is better | \n|---|---|---|---|---|\n| Hello | INT21 Rust | **15.11 ms** | **223.4 frames/s** | **0.0964** | \n| Hello | SGLang-Omni | 37.79 ms | 136.9 frames/s | 0.1573 | \n| Multiple sentences | INT21 Rust | **17.55 ms** | **249.2 frames/s** | **0.0864** | \n| Multiple sentences | SGLang-Omni | 38.61 ms | 154.9 frames/s | 0.1390 | \n| Voice reference | INT21 Rust | **23.58 ms** | **231.0 frames/s** | **0.0932** | \n| Voice reference | SGLang-Omni | 38.03 ms | 145.4 frames/s | 0.1481 | \n\nINT21 delivers 38–60% lower time to first audio and 1.59–1.63× the throughput across all three prompts. These are our B200 measurements, separate from SGLang’s published H200 results.\n\n## What the numbers mean\n\n- \n**Same GPUs, more speech.** In single-stream use, 1.59–1.63× throughput means 37–39% less GPU time per second of generated audio.\n- \n**Faster first response.** We deliver the first chunk of speech in 15–24 ms, versus about 38 ms for SGLang-Omni. In voice agents, that first-response delay is what users feel.\n- \n**Release-day support.** New model architectures usually require weeks of hand-porting before they run efficiently. The agent swarms closed that gap in an hour.\n- \n**Built for deployment.** Rust is compiled and memory-safe, with no Python interpreter in the serving path.\n\n## Faster inference means faster building\n\nWhen AI writes code, inference latency becomes development latency. Coding, debugging and revision loops all wait on model output. Faster inference lets builders test more ideas, fix problems sooner and ship faster. It also makes what they ship more responsive. Ultra-fast inference is a moat for building speed and user experience alike.\n\n## Steps to superintelligence\n\nThe bottleneck in AI is shifting from what models can do to how fast infrastructure can keep up. Self-improving infrastructure removes this hurdle, as AI agents generate, test, measure, and improve the software that runs AI models.\n\nAt INT21, we’ve achieved a major milestone: when a new architecture shipped, we only needed one hour for it to run faster than the officially supported runtime. Each engine that the agent swarms build makes the next one faster.\n\nThis is what progress toward superintelligence looks like. To learn more, explore INT21’s Inference Engine Factory [here](https://int21.ai/products/inference-engine-factory/), or contact us.", "url": "https://wpnews.pro/news/drama-3-shipped-one-hour-later-we-ran-it-63-faster-than-day-zero-support", "canonical_source": "https://int21.ai/insights/drama-3-shipped-one-hour-later-we-ran-it-63-percent-faster-than-day-zero-support/", "published_at": "2026-10-14 00:00:00+00:00", "updated_at": "2026-10-15 00:24:22.983107+00:00", "lang": "en", "topics": ["artificial-intelligence", "ai-infrastructure", "ai-agents", "large-language-models"], "entities": ["INT21", "Drama 3", "Fish Audio", "SGLang-Omni", "Inference Engine Factory", "B200", "Rust"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/drama-3-shipped-one-hour-later-we-ran-it-63-faster-than-day-zero-support", "markdown": "https://wpnews.pro/news/drama-3-shipped-one-hour-later-we-ran-it-63-faster-than-day-zero-support.md", "text": "https://wpnews.pro/news/drama-3-shipped-one-hour-later-we-ran-it-63-faster-than-day-zero-support.txt", "jsonld": "https://wpnews.pro/news/drama-3-shipped-one-hour-later-we-ran-it-63-faster-than-day-zero-support.jsonld"}}