{"slug": "open-source-voice-runtime-with-webrtc-and-semantic-endpointing", "title": "Open source voice runtime with WebRTC and semantic endpointing", "summary": "OpenLive, an independent open-source project, released version 26.7.15 of its model-neutral runtime for continuous voice agents, featuring WebRTC support, semantic endpointing, and a GPT-Live-comparable experience. The release includes Piper TTS, client-side audio intelligence, multi-agent sandbox, and durable profile/memory, aiming to provide a polished live voice UI without affiliation with OpenAI.", "body_md": "OpenLive is an open, model-neutral runtime for continuous voice agents. It separates deadline-sensitive interaction continuity from slower model cognition and preserves native duplex provider capabilities instead of forcing every model through a text-chat abstraction.\n\nImportant\n\n**Disclaimer:** This is an independent open-source project. It is **not an official OpenAI project** and has no affiliation with OpenAI.\n\n**Version 26.7.15** (`v26.7.15`\n\n) targets a **GPT-Live-comparable** experience: polished live voice UI, open neural speech (Piper), client-side audio intelligence, WebRTC session path, semantic endpointing, **real tools + multi-agent sandbox**, and durable profile/memory — with original visuals and model neutrality intact.\n\nFull parity matrix: [ docs/gpt-live-parity.md](/byte271/Openlive/blob/main/docs/gpt-live-parity.md) · Architecture roadmap:\n\n[· Open stack guide:](/byte271/Openlive/blob/main/docs/architecture-roadmap.md)\n\n`docs/architecture-roadmap.md`\n\n[· Credits:](/byte271/Openlive/blob/main/docs/open-source-stack.md)\n\n`docs/open-source-stack.md`\n\n[· Release notes:](/byte271/Openlive/blob/main/THIRD_PARTY_NOTICES.md)\n\n`THIRD_PARTY_NOTICES.md`\n\n`docs/release-26.7.15.md`\n\n**Voice surface**\n\n- Minimal black live surface + setup wizard; Live Presence theme tokens.\n- Full-screen voice presence with 11 named modes and multi-layer procedural orb.\n- Inline layout toggle, live dual transcript, conversation modes, speaking-style axes.\n- Push-to-talk, barge-in with local duck, camera/screen share affordances, visual cards.\n**Piper-first voice roster**(Lessac, Amy, Ryan, …) plus API-compatible fallbacks.- Brand chrome and package version:\n**26.7.15**.\n\n**Open AI voice**\n\n- Production path: cascade → OpenAI-compatible\n**Piper** TTS (via LocalAI, openedai-speech, or gateway-local Piper). - Gateway:\n`GET /v1/tts/status`\n\n,`POST /v1/tts/speak`\n\n; formant fallback for demos. - Licenses and attribution in\n`THIRD_PARTY_NOTICES.md`\n\n.\n\n**Client audio intelligence**\n\n- RNNoise-style noise suppression worklet (10 ms frames).\n- Silero-style VAD worklet + energy blend.\n- NLMS adaptive echo cancellation + windowed-sinc resampler.\n\n**Agent, tools & sandbox**\n\n- Internal agent (no OpenCode): search, deep research pool, calculator, time, identity, profile.\n- Path-safe sandbox file I/O + optional Chrome/Edge headless browse / screenshot / PDF.\n- Multi-agent pool (≤50) with SSE progress, agent classes, and destructive-action confirms.\n- Durable user profile (facts editor, drag-and-drop reorder) + session memory export.\n- See\n`sandbox/README.md`\n\nand`docs/architecture-roadmap.md`\n\n.\n\n**Transport, providers & tasks**\n\n- Binary WebSocket PCM +\n**gateway-native WebRTC**(DTLS data channels for events/PCM). - Provider-edge WebRTC (OpenAI Realtime SDP) when secrets are available.\n`POST /v1/webrtc/offer`\n\nanswers browser offers;`POST /v1/realtime/session`\n\nfor edge secrets.**Moshi** native duplex:`--provider moshi --moshi-url ws://127.0.0.1:8998/api/chat`\n\n.- Semantic endpointing (transcript-aware early end ~200 ms).\n- Visual cards + live translation demo (mock) / language-mode instructions.\n- Task lifecycle, evidence links, resume with dedup.\n- Configurable\n`--task-deadline-ms`\n\n. - Developer API:\n`GET /health`\n\n,`/v1/meta`\n\n,`/v1/sessions`\n\n,`/v1/agent/*`\n\n,`/v1/sandbox/*`\n\n,`/v1/profile`\n\n, MCP tools (+ optional API key). - Session persistence (JSONL under\n`data/openlive-sessions`\n\n), streaming safety holdback, MCP HTTP client.\n\n- Full RTP Opus media plane with packet FEC (data-channel PCM works today).\n- Official RNNoise WASM / Silero ONNX vendor weights (interfaces ready).\n- Transcript editing; production live-translation LLM hop.\n\n- Rust 1.83 or newer.\n- A modern Chromium, Firefox, or Safari browser.\n- Microphone permission.\n\n```\ncargo run -p openlive-gateway --release\n```\n\nOpen `http://127.0.0.1:8787`\n\nand select **Start** (or press `Space`\n\nin push-to-talk mode). The mock speaks with a lightweight formant voice so you can exercise barge-in, transcript, and tasks without external services.\n\nUse any OpenAI-compatible stack that exposes:\n\n`POST /v1/audio/transcriptions`\n\n`POST /v1/chat/completions`\n\n`POST /v1/audio/speech`\n\nwith`response_format: \"pcm\"`\n\n(24 kHz mono PCM16 preferred)\n\nExample with Piper-style voice ids:\n\n```\n# API keys: set in the environment only — never commit keys into this repo.\n# omit or leave empty for local unauthenticated servers\nexport OPENLIVE_MODEL_API_KEY\n\ncargo run -p openlive-gateway --release -- \\\n  --provider openai-compatible \\\n  --model-base-url http://127.0.0.1:8000/v1 \\\n  --asr-model whisper-1 \\\n  --llm-model your-chat-model \\\n  --tts-model tts-1 \\\n  --voice en_US-lessac-medium\n```\n\nSee [ docs/open-source-stack.md](/byte271/Openlive/blob/main/docs/open-source-stack.md) for LocalAI / openedai-speech / Piper wiring.\n\n```\n# Read the key from your shell environment (do not put keys in project files)\nexport OPENLIVE_MODEL_API_KEY\n\ncargo run -p openlive-gateway --release -- \\\n  --provider openai-realtime \\\n  --realtime-url wss://api.openai.com/v1/realtime \\\n  --realtime-model your-realtime-model \\\n  --voice alloy\ncargo run -p openlive-runtime --bin openlive-replay -- \\\n  --input fixtures/turn-completion.jsonl\n# Default: write session events/tasks under data/openlive-sessions\ncargo run -p openlive-gateway --release\n\n# Disable durability or safety:\ncargo run -p openlive-gateway --release -- --no-persist --safety false\n\n# Attach a remote MCP tool host:\ncargo run -p openlive-gateway --release -- --mcp-url http://127.0.0.1:3100/mcp\n\n# Deep model + local knowledge notes for complex turns:\ncargo run -p openlive-gateway --release -- \\\n  --provider openai-compatible \\\n  --model-base-url http://127.0.0.1:8000/v1 \\\n  --llm-model llama3.2 \\\n  --deep-llm-model qwen2.5-32b \\\n  --knowledge-dir ./knowledge\n\n# Hybrid: fast local duplex + deep cascade for hard turns\ncargo run -p openlive-gateway --release -- \\\n  --provider hybrid \\\n  --model-base-url http://127.0.0.1:8000/v1\n\n# Local Chronos full-duplex latency gate\ncargo run -p openlive-runtime --release --bin openlive-full-duplex-bench -- --turns 50\ncargo test --workspace --release\n# Integration tests need a debug binary:\ncargo build -p openlive-gateway && cargo test -p openlive-gateway --test task_lifecycle\nnode --test apps/openlive-gateway/web/tests/*.test.js\n```\n\nApache-2.0 for OpenLive source. Third-party speech stacks (Piper, etc.) have their own licenses — see `THIRD_PARTY_NOTICES.md`\n\n. Prefer running GPL TTS servers **out-of-process** over HTTP.", "url": "https://wpnews.pro/news/open-source-voice-runtime-with-webrtc-and-semantic-endpointing", "canonical_source": "https://github.com/byte271/Openlive", "published_at": "2026-07-16 05:11:07+00:00", "updated_at": "2026-07-16 05:25:06.037098+00:00", "lang": "en", "topics": ["artificial-intelligence"], "entities": ["OpenLive", "Piper", "WebRTC", "OpenAI", "GPT-Live", "RNNoise", "Silero", "Moshi"], "alternates": {"html": "https://wpnews.pro/news/open-source-voice-runtime-with-webrtc-and-semantic-endpointing", "markdown": "https://wpnews.pro/news/open-source-voice-runtime-with-webrtc-and-semantic-endpointing.md", "text": "https://wpnews.pro/news/open-source-voice-runtime-with-webrtc-and-semantic-endpointing.txt", "jsonld": "https://wpnews.pro/news/open-source-voice-runtime-with-webrtc-and-semantic-endpointing.jsonld"}}