YouTube is testing a new AI-powered tool to help make dubbing videos in non-English languages even easier—and at no cost.
YouTube teamed up with dubbing service Aloud, which is in turn a part of Google’s in-house incubator Area 120. With the addition of multi-language audio tracks to YouTube videos earlier this year, an AI-powered dubbing solution makes sense. YouTube creators previously needed to partner with a third-party dubbing service to provide audio tracks for different languages. But dubbing with Aloud allows creators to tap into multi-language markets at no additional cost.
The service works by providing a transcription for the creator before translating. Creators can review and edit that transcript before the dub is created, cutting out a lot of unnecessary work. YouTube is testing the tool with hundreds of creators, including the popular Mr. Beast. With enough dubbing hours under its belt, YouTube will open the feature to all creators. Currently Aloud is available in English, Spanish, and Portuguese.
YouTube’s VP of Creator Products Amjad Hanif says the company is working on technology to make the dub sound like the creator’s voice “with more expression and lip sync.” Generative AI in Aloud would allow creators access to better emotive transfer and lip reanimation to make video speakers seem like they are speaking the dub natively.
“Dubbing used to take weeks worth of effort and a large budget. But with ALoud, you only need a few minutes,” a blog post announcing the new technology reads. “We use advances in audio separation, machine translation and speech synthesis to reduce time-consuming and costly steps like translation, video editing, and audio production. You do not even need to know any other language than the one you already speak.”
“We want to make sure that synthetically dubbed content is never confused with the original, or with content that is dubbed by voice actors. So to ensure transparency, we require from our creators that all dubs produced by Aloud state clearly in the video description or in the post credits screen that they are synthetic dubs with respect to the original.”