Google’s AI can now create an “unlimited number” of soundtracks for video

Google’s AI research lab, DeepMind, has shared an update on the development of its video-to-audio (V2A) technology, a process that makes synchronised audiovisual generation possible.

According to Google’s AI page, V2A “combines video pixels with natural language text prompts to generate rich soundscapes for the on-screen action.” DeepMind says the technology can “understand raw pixels”, allowing for the real-time generation of video soundtracks.

READ MORE: EU AI Act explained: What does it mean for music producers and artists?

Additionally, the feature uses text prompts to generate rich audio soundtracks. Together, Google’s V2A technology supports the creation of dramatic scores, “realistic sound effects or dialogue that matches the characters and tone of a video.”

Google’s new AI feature means “enhanced creative control” meaning V2A technology can create an “unlimited number” of soundtracks.

While the tech company’s update on AI sounds creatively promising, there is still some progress to be made until the tool is rolled out to the general public.

The development of V2A is “improving”, however, it will be some time before the update will be widely available: “Still, there are a number of other limitations we’re trying to address and further research is underway,” the blog post reads.

The statement continues: “Since the quality of the audio output is dependent on the quality of the video input, artefacts or distortions in the video, which are outside the model’s training distribution, can lead to a noticeable drop in audio quality.”

Watch example clips of Google’s new V2A tool in action below.

In related news, Google recently came under fire from Sony Music, after the label raised suspicions that Google — along with OpenAI, Microsoft and over 700 other tech firms — have made “unauthorised uses” of Sony’s music catalogue to train AI. The public letter was titled ‘Declaration of AI Training Opt-Out’, and reads:

“We support artists and songwriters taking the lead in embracing new technologies in support of their art. Evolutions in technology have frequently shifted the course of creative industries. AI will likely continue that long-standing trend.

“However, that innovation must ensure that songwriters’ and recording artists’ rights, including copyrights, are respected. For that reason, SMG’s affiliates, Sony Music Publishing (SMP) and Sony Music Entertainment (SME), on behalf of themselves and their wholly owned or controlled affiliates, are making this affirmative, public declaration confirming that, except as specifically and explicitly authorized by either SME or SMP, as the case may be, each of them expressly prohibits and opts out of any text or data mining, web scraping or similar reproductions, extractions or uses.”

Learn more music technology news.