Meta releases AudioCraft, a generative AI tool that creates music from text prompts

Meta (earlier known as Facebook) has released a new open-source AI tool called AudioCraft. This generative AI tool enables users to create music through text prompts.

The tool bundles three generative AI models – AudioGen, EnCodec, and MusicGen.
MusicGen is a tool that uses text inputs to generate music. It was trained using over 20,000 hours of music that is either owned by Meta or licensed for this specific purpose. Meta’s EnCodec decoder helps users create sounds with fewer artefacts, preventing audio manipulation from causing distortion. AudioGen, on the other hand, creates audio based on written prompts, such as simulating the sound of barking dogs or footsteps. It was trained on public sound effects. a

Meta says that while AI-produced images and text have gained popularity, sound has not entirely caught up yet. Previous sound projects have been complex and often inaccessible to many, as per the company. The new toolkit aims to allow creators to customise their models and push the boundaries of what is possible, and the company is also open-sourcing these models to researchers, allowing them to train these models as their own with their datasets.
“AudioCraft works for music, sound, compression, and generation — all in the same place,” said Meta in a blog post announcing the tool. The company says that one of the advantages of this AudioCraft is its ease of use and reusability. Therefore, individuals who are interested in developing better sound generators, compression algorithms, or music generators can use this tool and enhance it by building on top of what others have already accomplished.
However, AudioCraft is not intended for regular users as it requires technical skills to use the tool effectively. It is primarily designed for research purposes, according to the company. The developers are currently working on enhancing the performance and control methods of these models to expand their capabilities.
Recently, Google also released its text-to-music tool, MusicLM, which works in a similar way, creating music from text prompts.