Meta Platforms, the company formerly known as Facebook, recently released a new AI research tool called AudioCraft. Meta’s AudioCraft contains several pre-trained models those are MusicGen, AudioGen, and EnCodec. While AudioGen, which is trained on public sound effects, generates audio from text-based user inputs, MusicGen, which was trained with Meta-owned and expressly licensed music, produces music from such inputs and an improved version of EnCodec decoder, which allows for higher quality music generation with fewer artifacts
Voice conversion aims to modify an audio recording to change attributes like a speaker’s gender, accent or age. Text-to-speech generation models in Audiocraft can synthesize completely artificial speech from text. With further improvements, this technology could enable more realistic virtual assistants and AI voice actors.
What Is Meta AudioCraft?
By releasing Audiocraft as an open research tool, Meta hopes to facilitate innovation and progress in audio and speech processing. The company’s previous deep learning research projects like ConvNetJS and Caffe2 have benefited the broader AI research community. Audiocraft has the potential to spur new developments in accessibility tools, virtual reality experiences and smart home devices.
In order to allow researchers and professionals to train their own models using their own datasets and advance the field of AI-generated audio and music, Meta has open-sourced these models. Any type of high-fidelity audio production demands the modeling of intricate signals and patterns at various scales. Since it consists of both local and long-range patterns, ranging from a set of notes to a global musical structure with several instruments, music is possibly the most difficult sort of audio to produce.
From Text To Audio
With AudioGen, Meta showed that it is possible to teach AI models to generate audio from text. The model can produce the environmental sound that corresponds to an acoustic scene given a written description, with realistic recording settings and intricate scene context.
The AudioCraft model can produce high-quality music with continued consistency and is simple to use thanks to a friendly interface. In comparison to earlier research in the field, Meta have simplified the design of generative models for audio with AudioCraft. AudiCraft is providing users with the full recipe to experiment with the models that Meta has been building over the past few years, as well as the freedom to push the boundaries and create their own models.
The same program, AudioCraft, can be used for sound production, compression, and music. People who want to create better sound generators, compression algorithms, or music generators can do so in the same code base because it is simple to build upon and reuse.
Also Read | Nextgen Smart Ring: Introducing Noise Luna Ring
Why AudioCraft Is Open Source?
For the benefit of the larger community, Meta has made their audio research framework and training code available under the MIT license. Meta also thinks that by creating more sophisticated controls, such models can benefit both professionals and amateur musicians.
The idea put forth by Meta is that having a strong open source base would encourage creativity and enhance how we create and consume audio and music in the future. Imagine beautiful bedtime readings with sound effects and dramatic music. Meta believes that MusicGen can evolve into a new kind of instrument with greater controls, much like synthesizers did when they originally arrived.
