Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

They stole my voice with AI

jeffgeerling.com

Post reply on HN
Print view View original

161–170 of 461 posts

Re: They stole my voice with AI

#161

Post by Eisenstein »

pjc50 wrote: post #154
vasco wrote: post #138

Earlier quoted context omitted.

The best we can hope for is that one personally avoids this for the first 5 years or so, and then it gets so widespread and easy that everyone will start doubting any videos they watch. The same way it took social media like reddit a few years of "finding the culprit" / "name and shame" till mods figured out that many times the online mob gets it wrong and so now that is usually not allowed. But many people will suff…

> everyone will start doubting any videos they watch. This kills the medium. Just as ubiquitous scam calls have moved people away from phones, this moves people away from using media which cannot be trusted. Done enough this destroys reporting and therefore democracy. I wonder when the first nonexistent candidate will be elected.

It is inevitable at this point that video as 'proof' will be killed. That we cannot do reporting and it destroying democracy is a little too much 'sky is falling'. Democracy existed before video.

Re: They stole my voice with AI

#162

Post by kmlx »

ryzvonusef wrote: post #135

Everyone has their own fears about AI, but my fears are especially chilling; what if AI was used to imitate a person saying something blasphemeous? My country is already has blasphemy lynching mobs based on the slightest perceived insult, real or imagined. They will mob you, lynch you, burn your corpse, then distribute sweets while you family hide and issue video messages denouncing you and forgiving the mob. And thi…

> what if AI was used to imitate a person saying something blasphemeous?

> My country is already has blasphemy lynching mobs

in your case the problem is not AI, it’s your country.

Re: They stole my voice with AI

#163

Post by moffkalast »

Yeah it's pretty much an inevitability that anyone who's ever posted more than 10 seconds of their voice will have it stolen eventually if anyone has a motive to do it. That's already all it takes to few shot a decent TTS _today_.

Most likely all existing youtubers will have complete voice and video digital clones made out of them. Then you can also tune an LLM on their scripts and it'll respond in the same character as well.

In theory you could also bring back ones who are dead, which would be very interesting in a historical sense. Like if we had hundreds of hours of Napoleon talking in front of a camera, it would be trivial to recreate a digital version of him for anthropologic study, maybe even having various figures debate things with each other. That's what historians a century later after we all die will be able to do with impunity.

Re: They stole my voice with AI

#164

Post by phs318u »

microtonal wrote: post #150
adityaathalye wrote: post #89

If LLMs are the ultimate remix machine, then is anyone with a RAG a digital DJ? One can't help but wonder what theft even means any more, when it comes to digital information. With the (lack of) legal precedent, it feels like the wild wild west of intellectual property and copyright law. Like, if even a superstar like Scarlett Johansson can only write a pained letter about OpenAI's hustle to mimic her "Her" persona,…

what can the comparatively garden-variety niche nerd do? [...] Like Geerling, feel equally sad / angry / frustrated For this kind of misuse, the person needs to have some fame, or it's not interesting to steal their voice. In such cases, their fame can be used for retribution. E.g. I can't imagine that this will be good for the reputation of Elecrow in the end. Next time I read the name of this company, I'll think oh…

> For this kind of misuse, the person needs to have some fame, or it's not interesting to steal their voice

This can still be very useful when used against non-famous people e.g. in a bitter custody dispute by one party to besmirch the other.

Re: They stole my voice with AI

#165

Post by sureglymop »

sandreas wrote: post #160

This is exactly the reason why I'm not open sourcing a tool I developed where you can take an audio book together with an epub to build an ljspeech dataset and train a voice model. Although it was not too hard to create I believe making it easier is something i don't like to achieve... I hate to say this but ruining a narrators existence with AI seems to get easier every day.

I do think that the floodgates are open now. AI is still absolutely terrible to mediocre at coding at best but for more trivial tasks like this it'll get indistinguishably good AND businesses are ready to sell before even beginning to think of the consequences.

Re: They stole my voice with AI

#166

Post by blueflow »

ryzvonusef wrote: post #135

Everyone has their own fears about AI, but my fears are especially chilling; what if AI was used to imitate a person saying something blasphemeous? My country is already has blasphemy lynching mobs based on the slightest perceived insult, real or imagined. They will mob you, lynch you, burn your corpse, then distribute sweets while you family hide and issue video messages denouncing you and forgiving the mob. And thi…

> And we have no tools to control this.

Do you know "The boy who cried wolf"? Fabricate some allegations yourself and this will train people to disbelieve them.

Re: They stole my voice with AI

#167

Post by latexr »

kmlx wrote: post #162
ryzvonusef wrote: post #135

Everyone has their own fears about AI, but my fears are especially chilling; what if AI was used to imitate a person saying something blasphemeous? My country is already has blasphemy lynching mobs based on the slightest perceived insult, real or imagined. They will mob you, lynch you, burn your corpse, then distribute sweets while you family hide and issue video messages denouncing you and forgiving the mob. And thi…

> what if AI was used to imitate a person saying something blasphemeous? > My country is already has blasphemy lynching mobs in your case the problem is not AI, it’s your country.

The comment didn’t say the problem was AI, it said they feared its consequences, which is a perfectly valid concern.

It’s like if someone said “I’m scared of someone bringing a semi-automatic weapon to my school and doing a mass shooting. My country has lax laws about guns and their proper use”. And then you said “in your case the problem is not guns, it’s your country”.

I mean, it’s technically true, but also unhelpful. Such ingrained laws are hard to change and you can be placed in danger for even trying.

Before someone decries the gun example as not being comparable, it is possible to live in a country with a monumental number of guns and not have mass murdering every day. It’s called Switzerland.

But let’s please stick to the subject of AI, which is what the thread is about. The gun example is the first analogy which came to mind, and analogies are never perfect, so it’s unproductive to nitpick the example. I don’t mean to shift the conversation from one contentious topic to another.

Re: They stole my voice with AI

#168

Post by BurningFrog »

pjc50 wrote: post #154
vasco wrote: post #138

Earlier quoted context omitted.

The best we can hope for is that one personally avoids this for the first 5 years or so, and then it gets so widespread and easy that everyone will start doubting any videos they watch. The same way it took social media like reddit a few years of "finding the culprit" / "name and shame" till mods figured out that many times the online mob gets it wrong and so now that is usually not allowed. But many people will suff…

> everyone will start doubting any videos they watch. This kills the medium. Just as ubiquitous scam calls have moved people away from phones, this moves people away from using media which cannot be trusted. Done enough this destroys reporting and therefore democracy. I wonder when the first nonexistent candidate will be elected.

We've had Photoshop for decades, and I still see pictures everywhere.

Re: They stole my voice with AI

#170

Post by ryzvonusef »

firtoz wrote: post #159
ryzvonusef wrote: post #135

Everyone has their own fears about AI, but my fears are especially chilling; what if AI was used to imitate a person saying something blasphemeous? My country is already has blasphemy lynching mobs based on the slightest perceived insult, real or imagined. They will mob you, lynch you, burn your corpse, then distribute sweets while you family hide and issue video messages denouncing you and forgiving the mob. And thi…

> You can say a lot of things about 'oh backward countries' but this will not stay there, this will spread I'm sorry, but this is a cope out. The "lynching from apparent cultural deviation" is something that needs to be moved on from. Developed countries do the same too to some extent, with "cancel culture" and such. There are ways to have progress in this, and, well, to feed someone's entrepreneurial spirit, it's on…

Indeed, if one were to post a AI video of someone saying some racial slur or otherwise verboten language, sure it won't get them killed, but given how unemployeable and pariah they would be, that would be a death by a thousand cuts.

But Blasphemy by whatever means, is one of the tools by which society sets certain boundries, and it's really hard to move away from a model that worked so 'well' for us since the first civiliations.

Post reply on HN