(Quotes condensed for clarity)

There’s always at least a little fanfare when Drake or The Weeknd put out new music. So it was no surprise when the two put out their first track together since 2013 that the track started to instantly see heavy rotation. The song? Heart On My Sleeve. The problem? It wasn’t created by Drake or The Weeknd at all.

Heart On My Sleeve was created using AI. An algorithm created Drake and The Weeknd’s vocals, not the artist themselves. Sort of like the audio version of a deep faked image. They aren’t the only two artists who have seen their voices imitated to create music. Fans of Travis Scott put out an entire album using an AI-generated replica of the rapper’s voice. Frank Ocean fans used AI to peddle songs under the guise of being authentic, unreleased music. Streaming services like Spotify have had to remove tens of thousands of AI tunes from its service.

How Does AI-Made Music Work?

But let’s rewind. How can software replicate an artist’s voice in the first place?

Tools like Boomy (which played a big role in those thousands of AI songs on Spotify) use generative AI to create replica music. If you’ve ever used ChatGPT or the photo app Lensa, you’ve encountered generative AI before. Generative AI allows you to train software using examples of something and then the application uses its training to produce its best guess at a similar something. Show an app a bunch of Picasso paintings, for example, and it’ll then take a stab at making its own Picasso. Give an AI tool a bunch of dog pictures and then it may be able to spot dog photos on its own. Present a tool with enough Frank Ocean songs and, you guessed it, the software can then fabricate a Frank Ocean song of its own.

Regardless, the technology here has become extremely capable. In January, Microsoft researchers introduced an AI model that could mimic a person’s voice with minimal training — all it needed was a three-second sample. Advancements like this are great for ALS victims who’ve lost their voice. Or terrible, when you consider spammers can more easily pose as our loved ones.

Regardless of pros and cons, generative AI has reached the world of music. So what happens when any musician can give their track a Drake verse? What happens when any artist can become The Weeknd in just a weekend?

Attack of the Clones

For one, you start to see a lot of replicas. We spoke with Justin Bernardez, a producer whose 2 million TikTok followers watch his videos to see just how exactly he can recreate the sound of a Migos song or a Travis Scott song. Instead of using an algorithm to imitate an artist’s sound, Justin does it the old-fashioned way — i.e. making the beat, writing and recording the vocals, sprinkling in the artist’s favorite ad-libs and tuning his voice to sound like them. The result is something like a Migos song not by the Migos.

AI can’t replace everything a producer like Justin does, but it can replace the hardest part — turning your voice into The Weeknd’s voice. So how does he feel about all this? In short, impressed. “I’ve been loving it,” Justin says. “The options are endless. It’s been cool to take my music and imagine, ‘What would it sound like if The Weeknd sang my song?’” ”

In some of his most recent videos, Justin has started to experiment with AI himself, having Michael Jackson sing Bruno Mars songs, for example. AI increases the scope of the types of replicas he can make. “It would be interesting to make singers rap and rappers sing,” says Justin. “Using AI, I could make Celine Dion rap. I could make Barack Obama sing.” It’s even opened the door to allow Justin to imitate more types of artists. “People have [always] asked ‘can you do a Rihanna song?’ and my voice can’t do what hers can do!” With AI, Justin can finally be Rihanna.

What Are Artists Doing In Response To AI-Music?

When ChatGPT first emerged, we saw some embrace and others denounce the tool. The same is true for AI-made music.

For one, Drake is tired of seeing himself on songs that he didn’t lend his voice to. The music labels don’t seem to like it much either. Universal Music Group responded to Heart On My Sleeve with takedown notices sent to sites like YouTube and Spotify. Heart On My Sleeve is just one of many. Spotify had to remove thousands of AI-made music from its service recently.

But some musicians are open to AI fueling our favorite tunes. An artist like Grimes, for example, is okay with fans making her music for her so long as she gets a 50% cut of the royalties.

AI-Made Music: What Comes Next?

Tracks like Heart On My Sleeve may be artificial but the 9 million views the song raked in before being taken down is real. AI-made music isn’t going anywhere and it changes both how fans indulge in music and how the pros create the tunes we tap our feet to. “I almost think recording studios have the chance of going obsolete,” says Justin Bernardez. “[For example,] would Drake need to go to a studio? Or could he just take a record from somebody that made a song that sounds like him in their bedroom and never even have to record it. Why would Drake have to go record it if it sounds exactly like him?”

We here at Mozilla always have questions about who’s working to create AI in a responsible, ethical way (check our Responsible AI Challenge and white paper on trustworthy AI for more info on that.) Those questions may, eventually, have to be answered about AI music. There are also legal questions we’ve never had to answer before, like should using someone else’s voice in your song without their knowledge of it even be allowed? Some will side-eye the idea of algorithms involved in the music process. Others will embrace it. Regardless of where you land, this may only be the beginning. “This is pretty new and it already sounds this good,” says Justin. “I think the bigger concern is how much better could this possibly get?”

AI-Generated Music — What It Is And What Could Come Next?

Written By: Xavier Harding

Edited By: Audrey Hingle, Carys Afoko, Innocent Nwani and Tracy Karuiki

Art: Shannon Zepeda

Video Produced By: Xavier Harding

Video Edited By: Candice Kirby