Meta launches AI tool that dubs and lip-syncs Reels in multiple languages

Meta has launched an AI-powered translation tool that automatically dubs and lip-syncs Reels videos in different languages, starting with English and Spanish translations. The feature, which was previewed at Meta Connect last year, aims to help creators expand their global reach by making their content accessible to international audiences while maintaining authentic voice characteristics and synchronized mouth movements.

What you should know: The tool requires creators to have either 1,000 Facebook followers or a public Instagram account to access the feature.

• At launch, Meta AI Translations only supports English-to-Spanish and Spanish-to-English conversions, though more languages will be added in the future.
• The feature works by mimicking the creator’s sound and tone to produce authentic-sounding dubbed audio, with optional lip-sync functionality that matches mouth movements to the translated speech.
• All AI-generated content is clearly labeled, and viewers can remove translations through the three-dot menu if they prefer the original version.

How it works: Creators can access the translation feature before publishing their Reels through a simple three-step process.

• Users tap “Translate your voice with Meta AI” and choose from three options: translate voice, add lip sync, or preview the result.
• The tool analyzes the original audio and generates translated versions that maintain the creator’s vocal characteristics and speaking style.
• Once published, translated Reels appear in viewers’ preferred languages automatically, with metrics broken down by language to help creators track performance across different audiences.

Technical limitations: The feature works best under specific conditions to ensure accurate output quality.

• Meta recommends using the tool for face-to-face camera videos with clear audio and minimal background noise.
• The system can handle dubbing for up to two speakers in a single video, but speakers shouldn’t overlap or talk simultaneously for optimal results.
• Creators can also upload up to 20 custom dubbed audio tracks through Meta Business Suite, though this option isn’t available if they’ve already used the AI translation feature.

The big picture: This launch represents Meta’s continued push into AI-powered content creation tools as social media platforms compete to help creators reach broader audiences.

• The feature addresses a significant barrier for content creators looking to expand internationally without the time and expense of manual translation and dubbing.
• By integrating advanced lip-sync technology, Meta is positioning itself ahead of competitors in the race to make cross-language content creation seamless and authentic.