Sophon-Realtime v1: Open Full-Duplex Voice With a 32B Brain and vLLM-Omni

Adventists AI released Sophon-Realtime v1, an open full-duplex voice stack published on Hugging Face under Apache-2.0 weights and code, built on a shared 32B model and served via vLLM-Omni. The model card reports VoiceBench 82.8, equal to GPT-4o-mini-Audio and below GPT-4o-Audio's 86.8, and above open full-duplex entries MiniCPM-o 4.5 (76.1), Freeze-Omni (55.2) and Moshi (29.5), plus 81.8% on 1,000 Big Bench Audio spoken reasoning questions and 48% Pass@1 on Full-Duplex-Bench v3 tool calls. Adventists ships three builds — Tag (~0.6 s first audio, 16 users per 80 GB GPU), Flash (0.31 s, 40 users) and E2E (~350M talker, ~0.25 s, 16 users) — with an optional OpenAI-compatible GPT-Live gateway on port 8780.

Sophon-Realtime v1 is a new open full-duplex voice stack from Adventists AI , published on Hugging Face https://huggingface.co/adventists-ai/Sophon-Realtime-v1?ref=explainx with Apache-2.0 weights and a GitHub serving repo https://github.com/adventists-ai/sophon-realtime?ref=explainx . Unlike a speech-to-text → LLM → TTS pipeline, it listens while you talk , barges in when you interrupt , and marks laughs, sighs, and stress in the text before a speech model performs them. For builders comparing 2026 voice agents, the release matters because it bundles benchmark tables , one-GPU economics , and a GPT-Live-shaped API in one package. explainx.ai recently covered Haru-4B https://explainx.ai/blog/haru-4b-open-full-duplex-speech-model-apache-qwen3-5-higurashi-2026 without public evals and Shizuku-Duplex https://explainx.ai/blog/shizuku-duplex-japanese-full-duplex-voice-80ms-interaction-background-2026 with Japanese-focused metrics. Sophon is the first open family in this wave to claim VoiceBench 82.8 beside MiniCPM, Moshi, and closed GPT-4o-audio baselines on the same card. TL;DR | Question | Answer | |---|---| | License | Apache-2.0 weights and code commercial use allowed on card | | Brain | Shared 32B model for listening, turn-taking, and reply drafting | | Builds | Tag expressive NVV , Flash density , E2E fastest first audio | | VoiceBench | 82.8 overall on card table open + full duplex | | Big Bench Audio | 81.8% spoken reasoning accuracy self-run, per card | | Tool calls | 48% Pass@1 on Full-Duplex-Bench v3, 100% response rate when tools needed | | Serving | vLLM-Omni for Tag/Flash; 16–40 users / 80 GB GPU depending on build | | GPT-Live | Optional live/ gateway on port 8780 with OpenAI-compatible WebSocket | | Try online | ModelScope Studio link on the Hugging Face page | Three builds, one brain Adventists ships one cognitive model and three speech renderers . The model card table https://huggingface.co/adventists-ai/Sophon-Realtime-v1?ref=explainx is the quickest map: | Build | How speech is made | Best when | First audio one user | Users / 80 GB GPU | |---|---|---|---|---| | Tag default | Reply text includes sound tags ; expressive speech model performs them | Companions, character demos, rich NVV | ~0.6 s | 16 | | Flash | Same tags, lighter speech model | Contact center scale, in-car | 0.31 s | 40 | | E2E | ~350M talker emits continuous voice vectors, no speech tokens | Fastest time-to-first-byte with NVV | ~0.25 s | 16 | NVV-SuperBench non-verbal vocalizations scores on the card: Tag hits 0.86 / 0.80 zh / en F1 , Flash 0.77 / 0.63 , E2E 0.85 / 0.75 . Tag and Flash share APIs so you can swap renderers without rewriting clients; E2E can run on its own engine or, since v1.1.4, on vLLM-Omni with the same surface. Sophon-Realtime v1 open full-duplex voice model benchmark table on the model card Image: Adventists AI https://huggingface.co/adventists-ai/Sophon-Realtime-v1?ref=explainx Benchmarks: what the card actually claims All numbers below come from the November 2026 model card unless noted. Treat vendor-run evals as directional until you reproduce them on your hardware and audio set. Conversation quality VoiceBench The card places Sophon-Realtime v1 at 82.8 , equal to GPT-4o-mini-Audio and below GPT-4o-Audio at 86.8 . Among open full-duplex entries in the same table, it ranks above MiniCPM-o 4.5 76.1 , Freeze-Omni 55.2 , and Moshi 29.5 . Spoken reasoning Big Bench Audio On 1,000 spoken reasoning questions , the card lists 81.8% accuracy for Sophon with open weights, versus 58% for Raon SpeechChat and 4% for Moshi. Sophon transcripts answers and grades with gpt-4.1 , per the footnote. Tool use under disfluent speech Full-Duplex-Bench v3 Here is the builder-relevant row: | Model | Pass@1 | Responds when it should | |---|---|---| | GPT-Realtime closed | 60% | 96% | | Gemini Live 3.1 | 54% | — | | Sophon-Realtime | 48% | 100% | | Grok | 43% | — | Sophon does not beat the closed frontier on raw Pass@1, but the card stresses it always responds when a tool call is appropriate—a failure mode that breaks real agents. Hearing tone paralinguistic Listening uses DuplexJev , Adventists’ open speech-decision model. Combined LISTEN + StressTest + MSPB score: 48.6 for DuplexJev, ahead of Gemini 3.8 Flash 46.9 and Qwen3-Omni 44.6 on the card. Two speech bubbles passing a turn-taking bead, illustrating full-duplex voice agent listening while speaking How the stack works mechanism, not marketing The card’s architecture section reads like a checklist of what Fish Audio Live 3 https://explainx.ai/blog/fish-audio-live-3-full-duplex-voice-agents-november-2026 and OpenAI GPT-Live https://explainx.ai/blog/gpt-live-1-api-openai-voice-agents-september-2026 buyers already ask for: 1. Continuous listening — at each pause the model emits a single token deciding whether you finished, continued, back-channeled, or paused to think. 2. Early drafting — it starts a reply at your first pause and only releases audio when it takes the floor; if you keep talking, the draft is discarded silently. 3. Tagged paralinguistics — the brain inserts laughter, sigh, breath, cough, emphasis markers; renderers perform them instead of reading tags aloud. 4. Coupled text and audio streams — transcripts stay available for logging, tools, and downstream LLMs. 5. Interrupt-safe memory — history keeps audio both sides heard; partial assistant audio is trimmed on barge-in. The E2E build replaces discrete speech tokens with continuous voice vectors from a ~350M talker—a design closer to research speech codecs than classic TTS chains. Quick start: Tag on one GPU The fastest path on the card: pip install "sophon-realtime trt " sophon-realtime serve Requirements called out in the README path: - Python 3.12 , Linux , CUDA 13 - trt extra required — without TensorRT the project warns the voice path is about 3× slower under load - First run downloads ~42 GB Tag weights, then listens on port 8000 GitHub scripts clone adventists-ai/sophon-realtime , run tag/install.sh , then GPU=0 PORT=8000 bash start.sh . Cold start can take 15–20 minutes the first time, 5–8 minutes later. WebSocket API: ws://HOST:8000/v1/realtime for duplex sessions; REST: POST /v1/chat/completions with audio in and text+audio out. Example client: python client example.py --wav my question.wav --lang en --voice ryan en . GPT-Live gateway For teams already running OpenAI-shaped clients: cd sophon-realtime/live && BUILD=tag PORT=8000 LIVE PORT=8780 bash start live.sh Clients point at ws://HOST:8780/v1/live/sessions . Sophon keeps voice; when a request needs lookup, the gateway can say a short filler, call your backend LLM , and speak results. The card claims pairing with DeepSeek V4 Pro lifts Big Bench Audio from 81.2 to ~83 in their configuration—verify on your stack before quoting to customers. Fine-tuning without retraining the voice Adventists documents LoRA fine-tuning on the thinker backbone via ms-swift , keeping audio encoder and voice frozen: pip install "ms-swift =4.6" git clone https://github.com/adventists-ai/sophon-realtime && cd sophon-realtime/finetune modelscope download --model AdventistsAI/Sophon-Realtime-v1 --include 'base/ ' 'adapters/ ' --local dir Sophon-Realtime-v1 Training continues the bundled tag LoRA so laughter and cough tags stay aligned. One 80 GB GPU is enough per the finetune README. Open-weight model pieces distributed to trays, echoing Apache-2.0 Sophon builds builders can self-host Comparisons: where Sophon fits in your stack | Need | Sophon angle | explainx.ai alternate | |---|---|---| | Smallest open duplex experiment | Haru-4B on ~12 GB, no public benchmarks | Haru-4B guide https://explainx.ai/blog/haru-4b-open-full-duplex-speech-model-apache-qwen3-5-higurashi-2026 | | Japanese companion, 80 ms ticks | Shizuku-Duplex | Shizuku post https://explainx.ai/blog/shizuku-duplex-japanese-full-duplex-voice-80ms-interaction-background-2026 | | Telephony latency SLOs | Commercial Tell-1 | Retell Tell-1 https://explainx.ai/blog/retell-tell-1-voice-model-contact-center-latency-2026 | | Closed GPT-Live SLA | OpenAI hosted | GPT-Live API https://explainx.ai/blog/gpt-live-1-api-openai-voice-agents-september-2026 | | Duplex + fish-scale agent demos | Fish stack | Fish Live 3 https://explainx.ai/blog/fish-audio-live-3-full-duplex-voice-agents-november-2026 | When to pick Sophon: you want Apache-2.0 , self-hosted duplex on one H100-class GPU , with published VoiceBench and tool-call numbers and a GPT-Live migration path . When to wait: you need sub-100 ms Japanese ticks Shizuku’s pitch , Mac-first MLX Haru’s path , or you cannot run CUDA 13 + TensorRT in production. What people are asking Are the benchmarks independent? VoiceBench and Full-Duplex-Bench rows mix official leaderboards with self-run Sophon jobs Big Bench Audio, some NVV runs . Budget time to replicate on your audio before shipping SLAs. Why three builds instead of one checkpoint? Speech quality and GPU multiplexing trade off. Flash sacrifices some NVV F1 in English 0.63 vs 0.80 on Tag to hit 40 users per GPU. Is DuplexJev separate weights? The card markets DuplexJev as the listening module inside Sophon; fine-tune docs focus on the thinker LLM only. ModelScope vs Hugging Face? Deploy weights are linked per build page; finetune downloads use ModelScope paths in the script while marketing lives on HF. HN traction? A Hacker News thread https://news.ycombinator.com/item?id=50434577&ref=explainx exists but had no substantive comments when we checked—hands-on reports may come later. Cost and capacity planning Capacity math on the card is blunt: one 80 GB GPU hosts 16 Tag users or 40 Flash users at recommended concurrency. If your contact center assumes P99 first audio under 500 ms , Flash’s 0.31 s cited first-audio figure is the build to model in load tests, not Tag’s ~0.6 s expressive path. Download and storage also belong in finance models: - ~42 GB Tag weights on first sophon-realtime serve - TensorRT path effectively mandatory for production multiplexing per README warnings - Finetune artifacts via ModelScope base/ and adapters/ bundles Pair Sophon with the same observability you would run on GPT-Live API pilots https://explainx.ai/blog/gpt-live-1-api-openai-voice-agents-september-2026 : log barge-in rate , tool-call success , and truncated assistant audio after interrupts. The card’s 100% response rate on Full-Duplex-Bench v3 matters only if your gateway preserves those semantics under real network jitter. Security and abuse notes for self-hosted voice Self-hosted duplex endpoints are attractive prompt-injection and social-engineering surfaces when wired to customer CRMs. Treat the live/ GPT-Live gateway like any other externally reachable WebSocket: mutual TLS, per-session API keys, rate limits, and separate tool sandboxes for backend LLMs. explainx.ai’s MCP explainer https://explainx.ai/blog/what-is-mcp-model-context-protocol-guide is a useful template for how tool handoffs should stay scoped when duplex models call backends mid-call. Limitations and licensing footnotes - Hardware gate: Without sophon-realtime trt , expect much slower speech under concurrent load. - Weight size: Tag download ~42 GB ; plan artifact mirrors and checksum policy like any frontier open model. - Third-party licenses: The card’s NOTICE file keeps component-level terms; read before redistributing bundles. - Business contact: jiejin@adventists.ai on the model page for commercial enquiries. - Paper link: The Hugging Face page references arXiv 2609.21967 NemotronLabs VoiceChat lineage for readers who want architecture context beyond the marketing tables. DuplexJev and the listening stack Sophon’s listening path is not a generic Whisper front-end. The card brands DuplexJev as a speech-decision model that feeds the 32B thinker. That split explains why paralinguistic understanding scores are reported separately from VoiceBench conversation quality: one measures ears , the other dialogue . If you are benchmarking against Qwen3-Omni https://explainx.ai/blog/qwen-3-8-omni-flash-launch-2026 or Gemini Live, reproduce LISTEN , StressTest , and MSPB tasks on your audio before claiming superiority—vendor F1 tables used Gemini 3 Flash as judge on NVV clips. Related reading Benchmark figures and install commands reflect the Adventists AI model card and GitHub README as accessed November 15, 2026. Re-run VoiceBench and latency tests on your own GPUs before production commitments.