Show HN: OpenTalking – Privately deployable real-time AI avatar Datascale AI released OpenTalking, an open-source orchestration framework for real-time digital-human conversations, covering frontend interaction, session state, LLM replies, STT, TTS, voice selection, interruption control, subtitle events, WebRTC audio/video playback, and local or remote model services. The framework supports private deployment with local STT/TTS, OpenAI-compatible LLMs, knowledge bases, memory, OmniRT remote inference, Docker, and distributed deployment, and offers a Mock mode for fast trials without downloading model weights. Open-source real-time digital-human pipeline: LLM, TTS, WebRTC, character voices, and pluggable model backends 中文 /datascale-ai/opentalking/blob/main/README.zh.md · Documentation https://datascale-ai.github.io/opentalking/latest/en/ · GitHub https://github.com/datascale-ai/opentalking Demos webui-and-demos · Deployment choose-a-deployment-path · Quickstart quickstart · Models supported-models · Roadmap progress-and-roadmap · Docs & Community documentation-and-community OpenTalking is an open-source orchestration framework for real-time digital-human conversations. It covers the core path of a digital-human conversational product : frontend interaction, session state, LLM replies, STT, TTS and voice selection, interruption control, subtitle events, WebRTC audio/video playback, and calls into local or remote model services. OpenTalking is designed as a practical digital-human production stack. The WebUI, avatar and voice asset libraries, knowledge bases, memory, multi-session state, LLM / STT / TTS providers, WebRTC playback, and model backends are organized in one project. You can start with the lightweight Mock mode, connect local QuickTalk / Wav2Lip, or use OmniRT for FlashTalk, FasterLivePortrait, and other higher-quality or more complex model workflows. Fast trial : mock / driverless mode , useful for validating the API, TTS, and WebRTC path before downloading video model weights. Real-time conversation : connect QuickTalk , Wav2Lip , FlashTalk , and other models for interactive digital-human dialogue. Video creation and cloning : reuse FasterLivePortrait runtime for audio/text-driven video creation and camera/uploaded-video-driven video clone workflows. Private deployment : supports local STT/TTS, OpenAI-compatible LLMs, knowledge bases, memory, OmniRT remote inference, Docker, and distributed deployment. More documentation: - Documentation site: https://datascale-ai.github.io/opentalking/latest/en/ https://datascale-ai.github.io/opentalking/latest/en/ - Chinese docs: https://datascale-ai.github.io/opentalking/latest/ https://datascale-ai.github.io/opentalking/latest/ OpenTalking provides a Web service interface for managing the digital-human conversation pipeline. You can select or create avatars, configure voices, LLM, TTS, STT, and digital-human driver models, inspect model connection status, and validate real-time conversation, subtitles, and audio/video playback on the same page. These demos cover three common frontend workflows: real-time conversation, video creation, and video clone. | Featured Product Scenarios | || |---|---|---| Healthcare guidance healthcare-zh.mp4 | Live commerce live-commerce.mp4 | Huangshan tourism guide huangshan-guide.mp4 | | A. Real-time Conversation | || |---|---|---| E-commerce livestream eCommerce.mp4 | Companion character companion.mp4 | News anchor newscaster.mp4 | | B. Video Creation | || |---|---|---| Audio driven audio drive.mp4 | Text driven text drive.mp4 | Cloned voice driven cloned voice drive.mp4 | | C. Video Clone | | |---|---| Realtime camera imitation camera drive.mp4 | Uploaded video imitation video drive.mp4 | OpenTalking's orchestration layer API / Worker / frontend and digital-human synthesis backend mock , local , direct ws , or OmniRT https://github.com/datascale-ai/omnirt can be deployed independently. If you are new to the project, start with Mock mode to validate the full path, then switch to a real rendering model based on your GPU, model, and private-deployment requirements. | Path | Recommended model / backend | Device reference | Best for | Details | |---|---|---|---|---| | Fast trial | mock | CPU / no GPU | Validate API, LLM, TTS, WebRTC, and browser playback without downloading model weights | | quicktalk / wav2lip QuickTalk https://datascale-ai.github.io/opentalking/latest/en/avatar models/deployment/quicktalk-local/ / Wav2Lip https://datascale-ai.github.io/opentalking/latest/en/avatar models/deployment/wav2lip-local/ quicktalk / wav2lip / musetalk Model and backend selection https://datascale-ai.github.io/opentalking/latest/en/model-support/selection/ sensevoice + local cosyvoice + quicktalk .venv , while CosyVoice runs in a dedicated sidecar venv Local STT/TTS + QuickTalk https://datascale-ai.github.io/opentalking/latest/en/recipes/local-quicktalk-audio/ flashtalk / flashhead / fasterliveportrait + OmniRT FlashTalk https://datascale-ai.github.io/opentalking/latest/en/avatar models/flashtalk/ / FasterLivePortrait https://datascale-ai.github.io/opentalking/latest/en/avatar models/fasterliveportrait/ Deployment https://datascale-ai.github.io/opentalking/latest/en/deployment/ Choose one of the two quickstart paths first: | Path | Use when | What you need | What it validates | |---|---|---|---| | Compshare image | You want to try OpenTalking before setting up dependencies or downloading model weights. | A Compshare instance created from the published image, with port 5173 open. | WebUI, LLM replies, streaming TTS, subtitle events, WebRTC delivery, and the prebuilt image workflow. | | Self deployment | You want to run the repo on your own machine or server, customize config, or continue into local/remote model deployment. | Python, Node.js, FFmpeg, .env provider config; real models also need GPU/runtime/model weights. | Mock first-run path, then local QuickTalk or remote OmniRT model paths. | If you want to try the OpenTalking + OmniRT + QuickTalk real-time digital-human path before setting up everything manually, use the community image we published on Compshare: - Image URL: image link https://www.compshare.cn/images/TdDwmKZUZebI?referral code=Hid5KUhcqlZEptmMEwKy2F - Guide: Compshare image quick experience https://datascale-ai.github.io/opentalking/latest/en/quick-start/ The image includes OpenTalking, OmniRT, the QuickTalk runtime environment, and model files. After deploying an instance, open port 5173 and visit the instance URL provided by the platform. If you need to restart services manually, follow the commands in the guide. Use this path when you want to run OpenTalking from source. Start with Mock mode if you do not want to download video model weights yet: Mock mode uses the built-in static frame, while LLM replies, streaming TTS, subtitle events, and WebRTC delivery still run through the full product path. git clone https://github.com/datascale-ai/opentalking.git cd opentalking uv sync --extra dev --python 3.11 source .venv/bin/activate cp .env.example .env Edit .env and configure at least an LLM. The default TTS can use the keyless edge voice. LLM, STT, and TTS are independent providers; see Configuration https://datascale-ai.github.io/opentalking/latest/en/reference/configuration/ and LLM / STT https://datascale-ai.github.io/opentalking/latest/en/speech models/llm-stt/ . bash scripts/start unified.sh --mock The default frontend URL is http://localhost:5173 . To specify ports: bash scripts/start unified.sh --mock --api-port 8210 --web-port 5280 Stop services: bash scripts/quickstart/stop all.sh After Mock mode works, choose a real model path based on your machine. Weight downloads, directory layout, mirrors, checks, and troubleshooting are maintained in the docs; the README keeps only the startup entrypoints: Local QuickTalk: consumer-GPU single-machine path export DIGITAL HUMAN HOME="${DIGITAL HUMAN HOME:-$HOME/digital-human}" export OPENTALKING MODEL ROOT="${OPENTALKING MODEL ROOT:-$DIGITAL HUMAN HOME/models}" export OPENTALKING TORCH DEVICE=cuda:0 export OPENTALKING QUICKTALK ASSET ROOT="$OPENTALKING MODEL ROOT/quicktalk" export OPENTALKING QUICKTALK WORKER CACHE=1 bash scripts/start unified.sh --backend local --model quicktalk --api-port 8210 --web-port 5280 Remote OmniRT / FlashTalk: high-quality or multi-card path bash scripts/start unified.sh \ --backend omnirt \ --model flashtalk \ --api-port 8210 \ --web-port 5280 \ --omnirt http://