Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

They stole my voice with AI

jeffgeerling.com

Post reply on HN
Print view View original

171–180 of 461 posts

Re: They stole my voice with AI

#171

Post by sureglymop »

ryzvonusef wrote: post #135

Everyone has their own fears about AI, but my fears are especially chilling; what if AI was used to imitate a person saying something blasphemeous? My country is already has blasphemy lynching mobs based on the slightest perceived insult, real or imagined. They will mob you, lynch you, burn your corpse, then distribute sweets while you family hide and issue video messages denouncing you and forgiving the mob. And thi…

My specific fear is that if a picture of you next to your name is available online, that becomes part of the training set of a future model. Paranoically, I do not have any picture of myself available online.

I could then trivially generate pictures or even videos of you e.g. by knowing your name. Of course that's just an example but I do think that's where we are headed and so the concept of "trust" will change a lot.

Re: They stole my voice with AI

#172

Post by 7bit »

kmlx wrote: post #162
ryzvonusef wrote: post #135

Everyone has their own fears about AI, but my fears are especially chilling; what if AI was used to imitate a person saying something blasphemeous? My country is already has blasphemy lynching mobs based on the slightest perceived insult, real or imagined. They will mob you, lynch you, burn your corpse, then distribute sweets while you family hide and issue video messages denouncing you and forgiving the mob. And thi…

> what if AI was used to imitate a person saying something blasphemeous? > My country is already has blasphemy lynching mobs in your case the problem is not AI, it’s your country.

That's a little too easy no? AI being used to imitate people definitely is a problem that needs to be addressed, and already is. Discarding that because there is a bigger issue is ignorant. Both can exist as a problem at the same time.

Re: They stole my voice with AI

#173

Post by arendtio »

XorNot wrote: post #130
arendtio wrote: post #54

Earlier quoted context omitted.

I am not convinced that it will be even 5 years. Have you tested elevenlabs[1]? They offer different voice cloning techniques today, starting from 30 seconds of audio input (sounds somewhat like the cloned voice but definitely not exactly the same) to multiple hours of voice input (sounds like the actual person). In addition, you can adjust the voices with a few parameters or simply create one by defining parameters.…

It's not the instant cloning that's the issue, it's cloning and tweaking - I don't think we quite have the methodologies built yet to optimize it. But we know it does matter - i.e. there's research which shows a good sound quality on a voice call improves whether people believe what you say[1]. Now in any individual session, you probably can't make particularly big alterations, but imagine say, Google or Amazon shipp…

So, you mean, similarly to how ad targeting works today, companies will optimize voices so that individuals respond accordingly to their targets?

Re: They stole my voice with AI

#174

Post by latexr »

blueflow wrote: post #166
ryzvonusef wrote: post #135

Everyone has their own fears about AI, but my fears are especially chilling; what if AI was used to imitate a person saying something blasphemeous? My country is already has blasphemy lynching mobs based on the slightest perceived insult, real or imagined. They will mob you, lynch you, burn your corpse, then distribute sweets while you family hide and issue video messages denouncing you and forgiving the mob. And thi…

> And we have no tools to control this. Do you know "The boy who cried wolf"? Fabricate some allegations yourself and this will train people to disbelieve them.

That sounds like a dangerous proposition. Either they fabricate allegations about a “nobody” and put them in trouble or they fabricate allegations about those in power and will be investigated and put themselves in danger. Neither strategy is good.

Re: They stole my voice with AI

#175

Post by underdeserver »

latexr wrote: post #167
kmlx wrote: post #162

Earlier quoted context omitted.

> what if AI was used to imitate a person saying something blasphemeous? > My country is already has blasphemy lynching mobs in your case the problem is not AI, it’s your country.

The comment didn’t say the problem was AI, it said they feared its consequences, which is a perfectly valid concern. It’s like if someone said “I’m scared of someone bringing a semi-automatic weapon to my school and doing a mass shooting. My country has lax laws about guns and their proper use”. And then you said “in your case the problem is not guns, it’s your country”. I mean, it’s technically true, but also unhelp…

Switzerland allows you to own a gun, but (generally) not to bear it. Huge difference.

Re: They stole my voice with AI

#176

Post by ryzvonusef »

kmlx wrote: post #162
ryzvonusef wrote: post #135

Everyone has their own fears about AI, but my fears are especially chilling; what if AI was used to imitate a person saying something blasphemeous? My country is already has blasphemy lynching mobs based on the slightest perceived insult, real or imagined. They will mob you, lynch you, burn your corpse, then distribute sweets while you family hide and issue video messages denouncing you and forgiving the mob. And thi…

> what if AI was used to imitate a person saying something blasphemeous? > My country is already has blasphemy lynching mobs in your case the problem is not AI, it’s your country.

Your country might not have lynching mobs, but you can't deny there are certain taboo topics in your society also, certain slurs and other opinions which would take you ages to clense and even then never fully.

If an AI fake-porn of some ordinary person involving a minor was unleashed, think of the utter shame and horror they would be treated by people for the rest of their lives, even if it were proven false.

No one would believe them, work with them, hire them, rent them, they would wish they had been lynched instead of the life they live.

Re: They stole my voice with AI

#177

Post by sandreas »

sureglymop wrote: post #165
sandreas wrote: post #160

This is exactly the reason why I'm not open sourcing a tool I developed where you can take an audio book together with an epub to build an ljspeech dataset and train a voice model. Although it was not too hard to create I believe making it easier is something i don't like to achieve... I hate to say this but ruining a narrators existence with AI seems to get easier every day.

I do think that the floodgates are open now. AI is still absolutely terrible to mediocre at coding at best but for more trivial tasks like this it'll get indistinguishably good AND businesses are ready to sell before even beginning to think of the consequences.

This. I'm still wondering why audio book giants like amazon/audible are not having license contracts with their narrators to do supportive narrating tasks by AI paying Them a fee for AI generating their voice. It probably would be win win...

Regarding how easy it was to clone my favorite narrators voice with open source tools I'm a bit afraid of what amazon could do with a whole cloud and massive man power

Re: They stole my voice with AI

#178

Post by latexr »

BurningFrog wrote: post #168
pjc50 wrote: post #154

Earlier quoted context omitted.

> everyone will start doubting any videos they watch. This kills the medium. Just as ubiquitous scam calls have moved people away from phones, this moves people away from using media which cannot be trusted. Done enough this destroys reporting and therefore democracy. I wonder when the first nonexistent candidate will be elected.

We've had Photoshop for decades, and I still see pictures everywhere.

Again this tired argument. Photoshop requires skill and time. AI generation takes a few seconds of typing some words. The scale is not comparable.

Re: They stole my voice with AI

#179

Post by vkou »

ConorSheehan1 wrote: post #93
oxygen_crisis wrote: post #82

Earlier quoted context omitted.

The court explicitly limited their decision to the voices of professional singers in that case: > ...these observations hold true of singing, especially singing by a singer of renown. The singer manifests herself in the song. To impersonate her voice is to pirate her identity... > We need not and do not go so far as to hold that every imitation of a voice to advertise merchandise is actionable. We hold only that when…

Doesn't this have an obvious edge case for every singer from now on though? If your voice is cloned before you become a singer of renown you have no protection.

Which is precisely why film producers are trying to get the power to do this to their actors.

They aren't going to use AI to have Tom Cruise in their film. He won't sign these rights away.

But they sure as hell want to have the next Tom Cruise sign those rights away as a condition for being hired to be Random Bystander #4 in a straight-to-streaming C-film.

Then, once he becomes successful and famous, they won't have to pay him a cent to keep using him, forever.

I can't wait for the future where even more of the wealth people who do work generate will be siphoned off to the owners.

Re: They stole my voice with AI

#180

Post by microtonal »

phs318u wrote: post #164
microtonal wrote: post #150

Earlier quoted context omitted.

what can the comparatively garden-variety niche nerd do? [...] Like Geerling, feel equally sad / angry / frustrated For this kind of misuse, the person needs to have some fame, or it's not interesting to steal their voice. In such cases, their fame can be used for retribution. E.g. I can't imagine that this will be good for the reputation of Elecrow in the end. Next time I read the name of this company, I'll think oh…

> For this kind of misuse, the person needs to have some fame, or it's not interesting to steal their voice This can still be very useful when used against non-famous people e.g. in a bitter custody dispute by one party to besmirch the other.

Yeah, which is what my second part is about :). The first part was about using a voice for e.g. promotion.
Post reply on HN