Voicebox by Meta

Voicebox by Meta AI is a groundbreaking generative AI model for speech. It boasts the unique ability to generalize to speech-generation tasks it wasn't specifically trained for, thanks to a novel approach based on Flow Matching. This enables Voicebox to learn from raw audio and accompanying transcriptions, allowing for unparalleled modification capabilities in any part of a given sample. The model sets new standards by outperforming existing models like VALL-E and YourTTS in intelligibility, audio similarity, and word error rate, all while being significantly faster. With over 50,000 hours of training data from public domain audiobooks in multiple languages, Voicebox excels in delivering high-quality, varied, and multilingual speech synthesis. While the model itself isn't publicly available to mitigate misuse risks, Meta provides extensive research materials and audio samples to showcase its potential. Voicebox opens a new frontier in speech generation, offering advancements in in-context text-to-speech synthesis, noise removal, cross-lingual style transfer, and more.