# Sophon-Realtime v1: Open Full-Duplex Voice With a 32B Brain and vLLM-Omni

> Source: <https://explainx.ai/blog/sophon-realtime-v1-open-full-duplex-voice-vllm-2026?utm_source=rss&utm_medium=feed&utm_campaign=blog>
> Published: 2026-11-15 00:00:00+00:00

**Sophon-Realtime v1** is a new **open full-duplex** voice stack from **Adventists AI**, published on [Hugging Face](https://huggingface.co/adventists-ai/Sophon-Realtime-v1?ref=explainx) with Apache-2.0 weights and a [GitHub serving repo](https://github.com/adventists-ai/sophon-realtime?ref=explainx). Unlike a speech-to-text → LLM → TTS pipeline, it **listens while you talk**, **barges in when you interrupt**, and **marks laughs, sighs, and stress** in the text before a speech model performs them.

For builders comparing 2026 voice agents, the release matters because it bundles **benchmark tables**, **one-GPU economics**, and a **GPT-Live-shaped API** in one package. explainx.ai recently covered [Haru-4B](https://explainx.ai/blog/haru-4b-open-full-duplex-speech-model-apache-qwen3-5-higurashi-2026) without public evals and [Shizuku-Duplex](https://explainx.ai/blog/shizuku-duplex-japanese-full-duplex-voice-80ms-interaction-background-2026) with Japanese-focused metrics. Sophon is the first open family in this wave to claim **VoiceBench 82.8** beside MiniCPM, Moshi, and closed GPT-4o-audio baselines on the same card.

## TL;DR

| Question | Answer | 
|---|---|
| License | **Apache-2.0** weights and code (commercial use allowed on card) | 
| Brain | Shared **32B** model for listening, turn-taking, and reply drafting | 
| Builds | **Tag** (expressive NVV),**Flash** (density),**E2E** (fastest first audio) | 
| VoiceBench | **82.8** overall on card table (open + full duplex) | 
| Big Bench Audio | **81.8%** spoken reasoning accuracy (self-run, per card) | 
| Tool calls | **48% Pass@1** on Full-Duplex-Bench v3,**100%** response rate when tools needed | 
| Serving | **vLLM-Omni** for Tag/Flash;**16–40 users / 80 GB GPU** depending on build | 
| GPT-Live | Optional **`live/` gateway** on port 8780 with OpenAI-compatible WebSocket | 
| Try online | ModelScope Studio link on the Hugging Face page | 

## Three builds, one brain

Adventists ships **one cognitive model** and three **speech renderers**. The [model card table](https://huggingface.co/adventists-ai/Sophon-Realtime-v1?ref=explainx) is the quickest map:

| Build | How speech is made | Best when | First audio (one user) | Users / 80 GB GPU | 
|---|---|---|---|---|
| **Tag** (default) | Reply text includes **sound tags** ; expressive speech model performs them | Companions, character demos, rich NVV | ~0.6 s | 16 | 
| **Flash** | Same tags, **lighter speech model** | Contact center scale, in-car | **0.31 s** | **40** | 
| **E2E** | **~350M talker** emits continuous voice vectors, no speech tokens | Fastest time-to-first-byte with NVV | ~0.25 s | 16 | 

**NVV-SuperBench** (non-verbal vocalizations) scores on the card: Tag hits **0.86 / 0.80** (zh / en F1), Flash **0.77 / 0.63**, E2E **0.85 / 0.75**. Tag and Flash share APIs so you can swap renderers without rewriting clients; E2E can run on its own engine or, since v1.1.4, on vLLM-Omni with the same surface.

Sophon-Realtime v1 open full-duplex voice model benchmark table on the model card
*Image: [Adventists AI](https://huggingface.co/adventists-ai/Sophon-Realtime-v1?ref=explainx)*

## Benchmarks: what the card actually claims

All numbers below come from the **November 2026 model card** unless noted. Treat vendor-run evals as directional until you reproduce them on your hardware and audio set.

### Conversation quality (VoiceBench)

The card places **Sophon-Realtime v1 at 82.8**, equal to **GPT-4o-mini-Audio** and below **GPT-4o-Audio at 86.8**. Among **open full-duplex** entries in the same table, it ranks above **MiniCPM-o 4.5 (76.1)**, **Freeze-Omni (55.2)**, and **Moshi (29.5)**.

### Spoken reasoning (Big Bench Audio)

On **1,000 spoken reasoning questions**, the card lists **81.8% accuracy** for Sophon with open weights, versus **58%** for Raon SpeechChat and **4%** for Moshi. Sophon transcripts answers and grades with **gpt-4.1**, per the footnote.

### Tool use under disfluent speech (Full-Duplex-Bench v3)

Here is the builder-relevant row:

| Model | Pass@1 | Responds when it should | 
|---|---|---|
| GPT-Realtime (closed) | 60% | 96% | 
| Gemini Live 3.1 | 54% | — | 
| **Sophon-Realtime** | **48%** | **100%** | 
| Grok | 43% | — | 

Sophon does **not** beat the closed frontier on raw Pass@1, but the card stresses it **always responds** when a tool call is appropriate—a failure mode that breaks real agents.

### Hearing tone (paralinguistic)

Listening uses **DuplexJev**, Adventists’ open speech-decision model. Combined **LISTEN + StressTest + MSPB** score: **48.6** for DuplexJev, ahead of **Gemini 3.8 Flash (46.9)** and **Qwen3-Omni (44.6)** on the card.

Two speech bubbles passing a turn-taking bead, illustrating full-duplex voice agent listening while speaking

## How the stack works (mechanism, not marketing)

The card’s architecture section reads like a checklist of what [Fish Audio Live 3](https://explainx.ai/blog/fish-audio-live-3-full-duplex-voice-agents-november-2026) and [OpenAI GPT-Live](https://explainx.ai/blog/gpt-live-1-api-openai-voice-agents-september-2026) buyers already ask for:

1. **Continuous listening** — at each pause the model emits a**single token** deciding whether you finished, continued, back-channeled, or paused to think.
2. **Early drafting** — it starts a reply at your first pause and only releases audio when it takes the floor; if you keep talking, the draft is discarded silently.
3. **Tagged paralinguistics** — the brain inserts**laughter, sigh, breath, cough, emphasis** markers; renderers perform them instead of reading tags aloud.
4. **Coupled text and audio streams** — transcripts stay available for logging, tools, and downstream LLMs.
5. **Interrupt-safe memory** — history keeps audio both sides heard; partial assistant audio is trimmed on barge-in.

The **E2E** build replaces discrete speech tokens with **continuous voice vectors** from a **~350M** talker—a design closer to research speech codecs than classic TTS chains.

## Quick start: Tag on one GPU

The fastest path on the card:

```
pip install "sophon-realtime[trt]"
sophon-realtime serve
```

Requirements called out in the README path:

- **Python 3.12** ,**Linux** ,**CUDA 13**
- **`[trt]` extra required** — without TensorRT the project warns the voice path is about**3× slower under load**
- First run downloads **~42 GB** Tag weights, then listens on**port 8000**

GitHub scripts clone **`adventists-ai/sophon-realtime`**, run `tag/install.sh`, then `GPU=0 PORT=8000 bash start.sh`. Cold start can take **15–20 minutes** the first time, **5–8 minutes** later.

**WebSocket API:** `ws://HOST:8000/v1/realtime` for duplex sessions; **REST:** `POST /v1/chat/completions` with audio in and text+audio out. Example client: `python client_example.py --wav my_question.wav --lang en --voice ryan_en`.

### GPT-Live gateway

For teams already running OpenAI-shaped clients:

```
cd sophon-realtime/live && BUILD=tag PORT=8000 LIVE_PORT=8780 bash start_live.sh
```

Clients point at `ws://HOST:8780/v1/live/sessions`. Sophon keeps voice; when a request needs lookup, the gateway can say a short filler, call your **backend LLM**, and speak results. The card claims pairing with **DeepSeek V4 Pro** lifts Big Bench Audio from **81.2 to ~83** in their configuration—verify on your stack before quoting to customers.

## Fine-tuning without retraining the voice

Adventists documents **LoRA fine-tuning** on the thinker backbone via **ms-swift**, keeping audio encoder and voice frozen:

```
pip install "ms-swift>=4.6"
git clone https://github.com/adventists-ai/sophon-realtime && cd sophon-realtime/finetune
modelscope download --model AdventistsAI/Sophon-Realtime-v1 --include 'base/*' 'adapters/*' --local_dir Sophon-Realtime-v1
```

Training continues the bundled **tag LoRA** so laughter and cough tags stay aligned. One **80 GB GPU** is enough per the finetune README.

Open-weight model pieces distributed to trays, echoing Apache-2.0 Sophon builds builders can self-host

## Comparisons: where Sophon fits in your stack

| Need | Sophon angle | explainx.ai alternate | 
|---|---|---|
| Smallest open duplex experiment | **Haru-4B** on ~12 GB, no public benchmarks | [Haru-4B guide](https://explainx.ai/blog/haru-4b-open-full-duplex-speech-model-apache-qwen3-5-higurashi-2026) | 
| Japanese companion, 80 ms ticks | **Shizuku-Duplex** | [Shizuku post](https://explainx.ai/blog/shizuku-duplex-japanese-full-duplex-voice-80ms-interaction-background-2026) | 
| Telephony latency SLOs | Commercial **Tell-1** | [Retell Tell-1](https://explainx.ai/blog/retell-tell-1-voice-model-contact-center-latency-2026) | 
| Closed GPT-Live SLA | OpenAI hosted | [GPT-Live API](https://explainx.ai/blog/gpt-live-1-api-openai-voice-agents-september-2026) | 
| Duplex + fish-scale agent demos | Fish stack | [Fish Live 3](https://explainx.ai/blog/fish-audio-live-3-full-duplex-voice-agents-november-2026) | 

**When to pick Sophon:** you want **Apache-2.0**, **self-hosted** duplex on **one H100-class GPU**, with **published** VoiceBench and tool-call numbers and a **GPT-Live migration path**.

**When to wait:** you need **sub-100 ms** Japanese ticks (Shizuku’s pitch), **Mac-first MLX** (Haru’s path), or you cannot run **CUDA 13 + TensorRT** in production.

## What people are asking

**Are the benchmarks independent?** VoiceBench and Full-Duplex-Bench rows mix **official leaderboards** with **self-run** Sophon jobs (Big Bench Audio, some NVV runs). Budget time to replicate on your audio before shipping SLAs.

**Why three builds instead of one checkpoint?** Speech quality and **GPU multiplexing** trade off. Flash sacrifices some NVV F1 in English (**0.63** vs **0.80** on Tag) to hit **40 users** per GPU.

**Is DuplexJev separate weights?** The card markets DuplexJev as the **listening** module inside Sophon; fine-tune docs focus on the **thinker LLM** only.

**ModelScope vs Hugging Face?** Deploy weights are linked per build page; finetune downloads use **ModelScope** paths in the script while marketing lives on HF.

**HN traction?** A [Hacker News thread](https://news.ycombinator.com/item?id=50434577&ref=explainx) exists but had **no substantive comments** when we checked—hands-on reports may come later.

## Cost and capacity planning

Capacity math on the card is blunt: one **80 GB** GPU hosts **16** Tag users or **40** Flash users at recommended concurrency. If your contact center assumes **P99 first audio under 500 ms**, Flash’s **0.31 s** cited first-audio figure is the build to model in load tests, not Tag’s **~0.6 s** expressive path.

Download and storage also belong in finance models:

- **~42 GB** Tag weights on first`sophon-realtime serve`
- **TensorRT** path effectively mandatory for production multiplexing per README warnings
- **Finetune artifacts** via ModelScope`base/*` and`adapters/*` bundles

Pair Sophon with the same observability you would run on [GPT-Live API pilots](https://explainx.ai/blog/gpt-live-1-api-openai-voice-agents-september-2026): log **barge-in rate**, **tool-call success**, and **truncated assistant audio** after interrupts. The card’s **100% response rate** on Full-Duplex-Bench v3 matters only if your gateway preserves those semantics under real network jitter.

### Security and abuse notes for self-hosted voice

Self-hosted duplex endpoints are attractive **prompt-injection and social-engineering surfaces** when wired to customer CRMs. Treat the **`live/` GPT-Live gateway** like any other externally reachable WebSocket: mutual TLS, per-session API keys, rate limits, and separate **tool sandboxes** for backend LLMs. explainx.ai’s [MCP explainer](https://explainx.ai/blog/what-is-mcp-model-context-protocol-guide) is a useful template for how tool handoffs should stay scoped when duplex models call backends mid-call.

## Limitations and licensing footnotes

- **Hardware gate:** Without**`sophon-realtime[trt]`** , expect much slower speech under concurrent load.
- **Weight size:** Tag download**~42 GB** ; plan artifact mirrors and checksum policy like any frontier open model.
- **Third-party licenses:** The card’s**NOTICE** file keeps component-level terms; read before redistributing bundles.
- **Business contact:**`jiejin@adventists.ai` on the model page for commercial enquiries.
- **Paper link:** The Hugging Face page references arXiv**2609.21967** (NemotronLabs VoiceChat lineage) for readers who want architecture context beyond the marketing tables.

## DuplexJev and the listening stack

Sophon’s **listening** path is not a generic Whisper front-end. The card brands **DuplexJev** as a speech-decision model that feeds the 32B thinker. That split explains why **paralinguistic understanding** scores are reported separately from **VoiceBench** conversation quality: one measures **ears**, the other **dialogue**.

If you are benchmarking against [Qwen3-Omni](https://explainx.ai/blog/qwen-3-8-omni-flash-launch-2026) or Gemini Live, reproduce **LISTEN**, **StressTest**, and **MSPB** tasks on your audio before claiming superiority—vendor F1 tables used **Gemini 3 Flash** as judge on NVV clips.

## Related reading

*Benchmark figures and install commands reflect the Adventists AI model card and GitHub README as accessed November 15, 2026. Re-run VoiceBench and latency tests on your own GPUs before production commitments.*