Amazon Is Using AI to Fix Terrible Dubbing on Its Streaming Shows

Amazon has launched AI and visual-effects lip-sync technology on Prime Video that synchronizes actors' mouth movements with dubbed audio, debuting on the German series Maxton Hall. The feature is already available on seasons one and two of Maxton Hall and will also apply to season three, which premieres on Dec. 9. Amazon says the process is "responsibly applied to deliver a better customer experience" and is intended to help creators' stories reach a broader global audience under creative oversight.

Dubbing can be useful for those who struggle to keep up with subtitles, but it's jarring when the actors' mouths don't line up with the audio. Amazon is looking to solve that problem on Prime Video by using AI and visual effects to synchronize https://www.aboutamazon.com/news/entertainment/prime-video-lip-sync-technology the dub with the actors' dialogue. This lip-sync technology will appear first on Maxton Hall before expanding to other titles. It's already available on seasons one and two of the popular German series, and will also work on season three /pc-games/69137/the-best-free-steam-games-for-2020?p=17 , which premieres on Dec. 9. "Syncing mouth movements to audio, it eliminates the disconnect that can occur with traditional dubbing," Amazon says. On Prime, searching Maxton Hall returned the original series and an option for "Maxton Hall - The World Between Us with Lip Sync for English Dub ." The actors' mouths do appear to line up with the English words in that version. Filmmakers might not like it, but Amazon argues the lip-sync tech will make it easier for creators' stories "to reach an even broader and global audience, all done under creative oversight to ensure the integrity of their artistic vision is preserved." "AI and VFX technologies power the process behind the scenes, responsibly applied to deliver a better customer experience," it adds. Last year, Amazon announced a pilot program https://www.aboutamazon.com/news/entertainment/prime-video-ai-dubbing-english-spanish for AI-aided dubbing on licensed movies and series that would not have been dubbed otherwise. And YouTube added multi-lingual dubbing /video-streaming-services/113077/want-to-go-global-youtube-adds-multilingual-dubbing last year.