New AI project, Riffusion, generates spectrograms to produce music
The technology is built upon a similar AI project which develops images from text.
Image: Smith Collection / Gado via Getty
A new AI project called Riffusion can generate spectrograms which produce sounds based on what style of music you ask it for.
The rise in AI over the last year has been phenomenal – we’ve seen Pilot Plugins’ futuristic AI avatars that can help you generate new MIDI and audio files, an AI cover of Dolly Parton’s Jolene, and more. A track released earlier this year featuring vocals by an AI has even surpassed 100 million streams.
As well as AI, more virtual and augmented reality technology has also made its way into the music industry. Now, following a huge spike in people generating AI portraits of themselves and various other images – Riffusion has found a way for this same technology to make music.
It works by building an indexed collection of spectrograms, each tagged with keywords that represent the style of music captured within the spectrogram. Once trained on this collection, Riffusion uses the same technology as Stable Diffusion, the image-generating AI, to make songs.
The project will generate a spectrogram image that has similar qualities to spectrograms that match a user’s text prompts. For example, if you were to ask it for “funky piano”, it will create a spectrogram similar to one that matches your prompt, and convert it to a funky piano-led track.
Check out the Riffusion demo below:
Back in October, Korean music business giant HYBE reportedly acquired a fake voice company for a whopping $32 million, suggesting artificial intelligence could be a prominent addition to the future of K-Pop music.
Whilst the Riffusion project is still in its early days, it’s likely further AI tech will continue to develop as time goes on. You can try out the tech yourself through Riffusions web app. For further details, go to riffusion.com.
