MARZ Unveils LipDub AI to Eliminate Mismatched Lip-Syncing of Dubbed Content for Global Distribution
Using AI technology to match facial movements to dubbed languages redefines the localization industry for content producers and advertisers
Emmy-nominated visual effects (VFX) studio and AI technology startup Monsters Aliens Robots Zombies (MARZ) today unveiled LipDub AI, its second AI for VFX product. LipDub AI is a state-of-the-art lip-syncing solution that dramatically improves the quality and appeal of dubbed content, and will transform the entertainment industry's localization sector.
LipDub AI is the world’s first completely automated lip-sync technology to achieve Hollywood-caliber results. In addition to delivering market-leading quality of visuals, LipDub is a scalable solution that visually transforms one minute of video footage into any language in seconds and in under 20 minutes of compute time for Hollywood-grade outputs.
The proprietary LipDub AI technology uses deep neural networks to seamlessly sync actors’ on-screen lip movements to any dubbed audio file, providing an elevated viewing experience to audiences worldwide watching non-native content. This groundbreaking approach to dubbing content is expected to help Hollywood studios and advertisers drive meaningful leaps in engagement and viewership. Moreover, it will enable creatives to create globally appealing content that truly transcends borders.
According to the MARZ executive leadership team, Jonathan Bronfman – Chief Executive Officer, Matt Panousis – Chief Operating Officer, and Lon Molnar – Chief Creative Officer, “The current standards for passable localization are far too low. Viewers around the world deserve to watch foreign-made content that looks great, just as studios and advertisers endeavor to create content that has global appeal. LipDub AI represents the next logical step in the evolution of dubbing, and it’s profoundly rewarding for MARZ to be spearheading that change.”
The LipDub research team is led by MARZ Chief Scientist and world-renowned professor at the Department of Computer Science at Tel-Aviv University, Danny Cohen-Or, an ACM fellow and The Isaias Nizri Chair in Visual Computing. He leads the MARZ AI research team alongside longtime collaborator Ali Mahdavi-Amiri, Assistant Professor at Simon Fraser University. Together they successfully conducted the research for Vanity AI, the first AI solution developed by MARZ. Vanity AI was launched in January 2023 and has already been used in over 45 Hollywood productions.
“With LipDub, we have succeeded in leveraging advanced generative AI to create photorealistic, lip-synced videos that are dynamic and temporally consistent,” said Cohen-Or.
For more information about LipDub AI visit LipDub.ai
About MARZ
Founded in August 2018, Monsters Aliens Robots Zombies (MARZ) is an Emmy-nominated VFX studio and AI technology start-up focused on premium entertainment verticals on a mission to help democratize VFX through the use of automated AI solutions. With its first two AI-driven solutions, Vanity AI and LipDub AI, and its world-class roster of clients, including Disney, Marvel, Netflix, Apple, Amazon Prime, HBO, Warner Bros, Showtime, Hulu, Paramount, CBS, and NBC Universal, MARZ continues to shape the future of visual effects and AI-driven creativity. For additional information, visit marzvfx.com.
Press Contact:
Lisa Ballard
Boulevard PR
lisa@boulevardpr.com
912-220-1179