Unauthorized voice cloning, an AI-powered Pandora's box that's causing more chaos than an overly energetic air guitarist at a karaoke bar. Amongst other things, it was quickly hijacked by scammers and turned into a nightmare for artists and the entertainment industry.
According to Michael Hasse, a Cybersecurity and Technology Consultant, voice cloning is pretty simple: "Individuals have a relatively limited range of pitches in normal speech, and their use of language will have a typical rhythm. Before LLMs, it took a lot of samples to get a good reproduction, but with modern systems, it can be done with a single sample of a few seconds as the LLM can match that against the known "library" and fill in the blanks quite accurately."
In other words, we're all just a bunch of predictable meat sacks, and AI has figured out how to mimic us with terrifying accuracy.
Sure, voice cloning has been used for some good causes, like helping introverts start podcasts or assisting those who have lost their voices for various reasons. But the dark side of this technology is starting to rear its ugly head.
Voice actors and musicians are finding themselves in the crosshairs of unauthorized cloning, with their voices being used in all sorts of unsavory contexts. Bev Standing, a voice actress, sued TikTok for allegedly using her voice without consent or compensation. Apparently, her sweet tones were used in videos featuring "foul and offensive language," which is a bit like finding out your angelic voice was used in a Quentin Tarantino film.
Scarlett Johansson accused OpenAI of using her voice for “Sky.” (This was not the first time she went after tech bros for using her voice.)
And it’s not just living artists feeling the pinch. The estate of George Carlin, the legendary comedian, sued Dudesy Media Company for producing an AI-generated comedy special mimicking Carlin’s voice and style. The unauthorized special, titled "George Carlin: I’m Glad I’m Dead," racked up nearly 500,000 views on YouTube, proving once again that the internet loves a good dumpster fire.
Even in the world of video games, AI voice cloning is causing a stir. Skyrim voice actors found their performances replicated in mods created for less-than-savory purposes. It’s one thing to slay dragons, but dealing with AI clones in dubious fan mods? That’s a quest no one signed up for.
As the lines between innovation and infringement become blurrier than a Monet painting, artists wonder how to protect their voices and who should be asked for help. In this article, we attempt to figure that out.
Content platforms: enablers or enforcers?
Entertainment industry leaders like YouTube, Spotify, and TikTok try to play sheriff to address the problem (as they should). Still, they often end up as bumbling deputies, with their policies being as effective as a chocolate teapot during British 5 o’clock tea.
At Spotify, the lines are about as clear as mud. According to Spotify's ruler, Daniel Ek, some AI-generated content gets the green light, but not all of it. Music that impersonates a real artist? Hell no. Auto-tune and Studio Sound? That’s totally fine. But there are also those AI-generated tunes that were "inspired" by an artist without going full copycat. Those are fine, too. And that's where we all are missing clarity.
TikTok is always eager to stay ahead. They're slapping an "AI-generated" label on watermarked content from third parties. It's a baby step, but hey, at least they're crawling in the right direction. But let's not forget that its effectiveness depends on user awareness, and we are talking about the platform where the most viral content includes people eating Tide Pods.
YouTube, the king of the video content hill, demands that creators label their AI-generated content "realistic." It’s a nice gesture, but enforcing it is another story. They’re essentially telling creators, "Please, be honest," but without an army of moderators and top-tier detection algorithms, it's a bit like asking cats to police a dog park. (And don’t get me started on Google’s AI advancements ‘cause it sucks).
But are these policies worth the digital paper they're written on? Just ask Drake. The rap king found himself in the AI hot seat not once but twice. First, an AI-generated track using his and The Weeknd's voices, "Heart on My Sleeve," blew up last year, racking up over 8.5 million views on TikTok before Universal Music Group threw a hissy fit and demanded its removal from all the platforms.
This year, Drake's back at it (but now not as a victim but as an offender) with an AI-generated Tupac track. The track vanished from his Instagram and X accounts after Tupac's estate threatened to sue him. But just like a bad penny, both songs keep turning up on YouTube and TikTok, no matter how hard someone tries to flush it. And that’s all you have to know about platforms and their rules.
There are even cases when social media giants damage instead of helping someone who was a victim of voice cloning. Take Erica Lindbeck, the voice of Futaba in Persona 5. Lindbeck was in a nightmare when an AI-generated video popped up, replicating her voice as Futaba belting out a Bo Burnham tune. She asked fans to report it to be removed and instead got criticized. The harassment got so bad that Lindbeck had to delete her Twitter/X account.
Not only content “hosters” companies are looking into setting up some ground rules, but AI companies as well. Here’s the take from the lawyer, Georgia Renard, Associate at XVII Degrees in Sydney, Australia: “Most AI companies have published policies, guidelines, charters of use, and codes of conduct that set out best-practice usage of their services. These have proven to be largely ineffective. As an example, OpenAI – the company behind ChatGPT and the as yet unreleased Voice Engine – has four universal usage policies that govern the use of its services. These are to:
1. Comply with applicable laws.
2. Not use [OpenAI’s] services to harm yourself or others.
3. Not repurpose or distribute output from [OpenAI’s] services to harm others
4. Respect [OpenAI’s] safeguards and safety mitigations.
"While these policies are all well and good to have, these sorts of best-practice usage policies are evidently not doing enough to prevent misuse. As far as responding to misuse, the most that OpenAI and comparable AI companies seem to be doing to address use that violates their policies is banning accounts. Banning accounts is, however, an inherently reactive step and does nothing to redress harm already caused. It also does nothing to prevent people with banned accounts from creating new accounts with different details.
"It is interesting to note that OpenAI has not released its AI voice cloning service 'Voice Engine' on the open market yet, deeming it 'too risky' for general release. This is not a strong vote of confidence in the capabilities of AI companies to prevent and regulate uses of their services that violate the law.”
And not only that. But as the case of Scarlett Johansson mentioned earlier in this article demonstrates, AI companies can’t even comply with their own rules. So what's the solution? If only I knew. But one thing's for sure: both content platforms and AI companies need to step up and do more.
Untangling All Things Legal
So, is there something you can do if someone cloned your voice? Should Kanye West and Joe Biden both silently suffer through numerous AI-generated songs using their voices plaguing YouTube?
We've asked legal experts how things are going. The main question to start with is, “Who owns the voice?”
According to Terry Quan, partner at FLG Foundation Law Group: “The voice of person comprises a part of a person’s right of publicity. As there is no federal law covering the right of publicity, the specific law for the right of publicity is based on statutes adopted on a state-by-state basis and based on common law (that is, based on case law). In California, the right of publicity is embodied in California Civil Code §3344 and protects a person’s name, voice, signature, photograph, and likeness and it enables a person to control their voice for their own economic benefit. So, in the case of a music artist, the music artist 'owns' their own voice as part of the music artist’s right of publicity.
"In the context of a record label with a contractual relationship with a musical artist, the appropriate question might be, 'Who owns the voice recording as part of a song?' This depends on what the contract says, and usually, the record label wants the broadest rights possible to publish and monetize a voice recording (along with the music). However, there are many copyrights involved in terms of the ownership of a song where the record label usually owns the 'master' (that is, the recorded performance of a song), and the performer/songwriter could own the copyright to the lyrics and music.
"It is possible that different record labels may have rights to different versions of a song performed by an artist, e.g., one label might own a studio recording, and one label might own a live performance. Each label would own its own version of that voice recording, and what the label could do with it would be governed by the respective contracts negotiated with the artist. Some artists, such as Taylor Swift, exercise a lot of control and may prevent the use of voice recordings as training material for generative AI platforms.”
According to John Michael Eden, Counsel at BurgherGray LLP, in many states, voice and likeness are not protected. “There aren't very clear laws stating you have a right of action against someone mimicking your voice and robbing you of the right to make money off your own content. There are, of course, exceptions for celebrities - for people who make a living from monetizing their likeness. There have been, however, some legislative changes in certain states. The Elvis Act in Tennessee, going into effect July 1st, 2024, gives an individual a right to protect their voices as a bona fide intellectual property right. The law covers both actual and simulated voices, and violations constitute a misdemeanor that can lead to criminal and civil penalties. The scope of the Elvis Act is pretty clear: You can't use someone's voice to publish, perform, distribute, or transmit it without their authorization.”
When it comes to the question of what a victim can do if their voice is cloned without permission, Danielle S. Van Lier, an experienced technology, IP, and entertainment attorney with a background in representing talent and expertise in the entertainment guilds and AIlays it out straight: "This question depends on the context and the jurisdiction. Focusing on the U.S., generally, the simple act of cloning the voice is unlikely to be considered a violation of the law; in most cases, the violation will come from how it is used. In most US states, laws like the right of publicity will provide some recourse for the victim, particularly if the voice is used in a commercial manner. Other laws, such as false light or defamation, might apply if the person is made to speak in a way that is false or objectionable."
But it's not just about the individual victim. Van Lier emphasizes the broader societal implications of unauthorized voice cloning: "It is becoming critical that we get this kind of legislation, not just because of the risks for the individuals whose voices or likenesses are used, but also to protect the people who hear or see them and might be misled. We have seen people scammed out of money by voice clones of their relatives, sometimes in pretty horrible ways, like claiming to have been kidnapped or in an accident. A robocall with a voice clone of President Biden urged people not to vote in the New Hampshire presidential primary election. There is a real danger from voice clones."
But how about our favorite entertainment industry? As explained to me by John Michael Eden, "Someone like Taylor Swift, U2, or Michael Jackson, when they do deals with music companies and music publishers, they often receive upfront fees and then a bonus structure based on how their work performs. In exchange, these artists sign over certain rights to their music. The specific recorded expression (often called the master copyright) is typically acquired by the music label, but in a more important sense, the artist still owns their voice. In other words, an artist doesn’t sell her “voice” to the music publisher. "
But what happens when an AI company partners with a label to train its model on licensed content, and the artist is against this use of their voice? According to Eden (who, by that point in our conversation, knew that my favorite artist was Daft Punk), it's a bit like the Wild West out there.
“To keep it even more simple: let's say I go out and spend a month pulling all of Daft Punk's work, and I take a couple of other adjacent experimental electronic bands and throw them in there, so there's like six bands in total, but Daft Punk makes up about 30% of the content I train my model on. So when you start hearing the product of my project, you're like, wow, this sounds so much like Daft Punk. Daft Punk (assuming they own the master copyrights to the work I borrowed) could go after me for using their music to train my model. The legal claim is simple: By taking in all of that music from Daft Punk, I had to create a temporary copy of it in my model, and that temporary copy violates the exclusive right of reproduction that copyright holders enjoy under US law. But once you get to the generative AI phase, where I'm releasing new stuff, the legal claims are hard to make. The reason that they’re challenging to make is that under existing federal copyright law it’s not clear that generative AI content is a derivative work. You see, copyright holders do have an exclusive right to make derivative works of their protected content. And that’s the challenge for Daft Punk. In our hypothetical, they’d have to show that we made a derivative work. That’s tough because our AI model ingested a lot of different music to create something that is not — if you look under the hood of the AI model— a derivative of any specific input into the system at the training phase. The generated content truly is new..."
And even though all those rules and nuances are different depending on geography. As Stefan Ellenberg, an entertainment and IP lawyer at Gutsch & Schlegel Rechtsanwälte i.P., puts it: "In cases of rights infringement in Germany, the violator must prove that they were authorized to use the rights. Whether the voice is identical or just highly similar is not legally decisive. Only when audible differences exist will a judge make an evaluation."
Georgia Renard, an Associate at XVII Degrees in Sydney, Australia, reminded us that there is no cohesive legal framework addressed specifically to the misuse of AI services to clone someone's voice without their permission: "There are no jurisdictions that have developed a comprehensive law or statute that deals specifically with the use—or misuse—of AI services in this way, at least not yet."
Zohaib Ahmed Founder & CEO of Resemble.ai, the leading AI voice generator that recently launched Rapid Voice Cloning (RVC), suggests another technology to solve the issue: "As we wait for legal standards to catch up, technologies like AI watermarking and deepfake detection can be implemented to trace the origin of content and hopefully prevent ownership disputes and misuse of AI."
So, where does that leave us? In a legal landscape with a patchwork of laws and regulations that may or may not apply depending on the jurisdiction and the specific circumstances of each case.
Moving forward: protecting voices in the digital age
"With great power comes great responsibility." And boy, do AI companies have a lot of power these days. But it's not about blocking AI technology altogether. It's about channeling it in a way that supports human creativity rather than replacing it. While creating laws and practices protecting it.
One step in the right direction is the recent agreement between SAG-AFTRA and Replica Studios. This deal allows actors to start selling their AI voice clones to game companies with industry-leading protections tailored to AI technology. It's a way for creators to safely explore new employment opportunities without getting totally exploited by the machines.
And surely federal laws to protect humans are coming, right?
However, let's not get too carried away just yet. As one of my favorite lawyers of all time, John Michael Eden, reminds us, getting Congress to pass appropriate legislation could be a bit like herding cats: "It's possible for laws to change to account for the economic loss to artists when synthetic content replaces demand for their work in the market. However, it will be challenging for Congress, as there are powerful lobbies on both sides - the entertainment industry in Hollywood and the AI industry in Northern California. The creative industry may be a bit overpowered now compared to the strengthening tech industry. Getting Congress to understand how the technology works and passing appropriate legislation will undoubtedly be difficult."
So, let's not get too cozy just yet. Instead, let’s think about the uncomfortable because the future is coming pretty fast: Will we see a future where the most popular artists are digital recreations, with their real counterparts pushed to obscurity? Will we lose something essential about what it means to be human in the process? And should we even allow AI to replicate human voices, or is there an ethical line that technology should never cross?
