How to Dub a Video With Two Speakers Into Another Language and Keep Each Voice (Open Source)

The miragea-voice project released miragea-dub, an MIT-licensed open-source command-line tool that dubs multi-speaker video into another language while preserving each speaker's cloned voice. The pipeline transcribes the clip, regroups fixed-width subtitle cues into whole sentences, clusters speakers without a GPU, translates each line to fit its original time slot, and renders per-speaker voices via the Miragea voice API, speeding lines up to 1.35x and flagging any that still overrun. It runs as a CLI, an MCP server, or an Agent Skill, and outputs a dubbed MP4, a separate audio track, and SRT/VTT subtitles.

Dubbing a video into another language is easy when one person talks. It gets hard the moment there are two speakers: a single synthetic voice for everyone makes interviews and podcasts unwatchable, and the translated sentences rarely fit the original timing. This tutorial dubs a real two-speaker clip from English into Spanish with an open-source tool, miragea-dub https://codeberg.org/miragea-voice/miragea-dub MIT . Each speaker keeps their own cloned voice, the lines are fitted into the original timing, and you get a dubbed MP4, an audio track and subtitles. AI video dubbing replaces the speech in a video with translated speech. A multi-speaker dub does four jobs: it transcribes the video, separates the speakers , translates each line so it can be spoken in the same time slot, and voices each speaker with a voice that resembles the original. miragea-dub runs those steps from the command line, as an MCP server for AI agents, or as an Agent Skill. Voices come from the Miragea https://miragea.ai voice API, so no GPU is needed. Both voices were cloned from the clip itself. Transcribing the Spanish track again returns the translated script almost word for word. pipx install miragea-dub needs ffmpeg export MIRAGEA API KEY=... key from https://miragea.ai/workspace/api miragea-dub doctor miragea-dub demo optional: offline run with test tones miragea-dub prepare two-voices.mp4 --to es --speakers 2 This writes two-voices.dub/dub.json . Miragea's speech recognition returns subtitle cues cut at a fixed width, sometimes mid-word, so miragea-dub regroups them into whole sentences using the punctuated full text. A built-in, GPU-free clustering step then labels who speaks S1, S2 . On the demo clip all six lines were assigned to the right speaker. Each line has a length budget. Fill the translation fields yourself, let Claude or Cursor do it through the MCP server, or run miragea-dub translate --translator openai with any OpenAI-compatible endpoint. Short lines matter: the dub has to fit. miragea-dub render two-voices.dub/dub.json --model expressive For each speaker miragea-dub cuts a 8-20 second reference from the video, hosts it on a temporary link and synthesizes every line with that voice. Lines are sped up slightly limit 1.35x to fit; anything that still does not fit is listed in lines to shorten . You get two-voices.es.mp4 dub + original audio + subtitles , two-voices.es.m4a for YouTube's additional-audio feature, SRT/VTT subtitles and a JSON report. { "mcpServers": { "miragea-dub": { "command": "uvx", "args": "--from", "miragea-dub", "miragea-dub-mcp" , "env": { "MIRAGEA API KEY": "your-key" } } } } Or install the video-dubbing skill from miragea-skills https://codeberg.org/miragea-voice/miragea-skills . The agent does the translation, so no second LLM key is needed. Can I dub a two-person interview and keep both voices? Yes, each speaker is cloned from their own lines. Is there an open-source alternative to hosted dubbing apps? The pipeline is MIT; voices need a Miragea API key. Bigger GUI projects such as VideoLingo and pyvideotrans exist too, this one is the small scriptable one for agents. Only clone voices you have permission to clone. Source: codeberg.org/miragea-voice/miragea-dub https://codeberg.org/miragea-voice/miragea-dub · PyPI https://pypi.org/project/miragea-dub/