Dubbing a video into another language is easy when one person talks. It gets hard the moment there are two speakers: a single synthetic voice for everyone makes interviews and podcasts unwatchable, and the translated sentences rarely fit the original timing.
This tutorial dubs a real two-speaker clip from English into Spanish with an open-source tool, miragea-dub (MIT). Each speaker keeps their own cloned voice, the lines are fitted into the original timing, and you get a dubbed MP4, an audio track and subtitles.
AI video dubbing replaces the speech in a video with translated speech. A multi-speaker dub does four jobs: it transcribes the video, separates the speakers, translates each line so it can be spoken in the same time slot, and voices each speaker with a voice that resembles the original. miragea-dub runs those steps from the command line, as an MCP server for AI agents, or as an Agent Skill. Voices come from the Miragea voice API, so no GPU is needed.
Both voices were cloned from the clip itself. Transcribing the Spanish track again returns the translated script almost word for word.
pipx install miragea-dub # needs ffmpeg
export MIRAGEA_API_KEY=... # key from https://miragea.ai/workspace/api
miragea-dub doctor
miragea-dub demo # optional: offline run with test tones
miragea-dub prepare two-voices.mp4 --to es --speakers 2
This writes two-voices.dub/dub.json. Miragea's speech recognition returns subtitle cues cut at a fixed width, sometimes mid-word, so miragea-dub regroups them into whole sentences using the punctuated full text. A built-in, GPU-free clustering step then labels who speaks (S1, S2). On the demo clip all six lines were assigned to the right speaker.
Each line has a length budget. Fill the translation fields yourself, let Claude or Cursor do it through the MCP server, or run miragea-dub translate --translator openai with any OpenAI-compatible endpoint. Short lines matter: the dub has to fit.
miragea-dub render two-voices.dub/dub.json --model expressive
For each speaker miragea-dub cuts a 8-20 second reference from the video, hosts it on a temporary link and synthesizes every line with that voice. Lines are sped up slightly (limit 1.35x) to fit; anything that still does not fit is listed in lines_to_shorten.
You get two-voices.es.mp4 (dub + original audio + subtitles), two-voices.es.m4a for YouTube's additional-audio feature, SRT/VTT subtitles and a JSON report.
{ "mcpServers": { "miragea-dub": { "command": "uvx", "args": ["--from", "miragea-dub", "miragea-dub-mcp"],
"env": { "MIRAGEA_API_KEY": "your-key" } } } }
Or install the video-dubbing skill from miragea-skills. The agent does the translation, so no second LLM key is needed.
Can I dub a two-person interview and keep both voices? Yes, each speaker is cloned from their own lines.
Is there an open-source alternative to hosted dubbing apps? The pipeline is MIT; voices need a Miragea API key. Bigger GUI projects such as VideoLingo and pyvideotrans exist too, this one is the small scriptable one for agents.
Only clone voices you have permission to clone.
Source: codeberg.org/miragea-voice/miragea-dub · PyPI