Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

They stole my voice with AI

jeffgeerling.com

Post reply on HN
Print view View original

241–250 of 461 posts

Re: They stole my voice with AI

#241

Post by shagymoe »

cynicalsecurity wrote: post #228
ryzvonusef wrote: post #135

Everyone has their own fears about AI, but my fears are especially chilling; what if AI was used to imitate a person saying something blasphemeous? My country is already has blasphemy lynching mobs based on the slightest perceived insult, real or imagined. They will mob you, lynch you, burn your corpse, then distribute sweets while you family hide and issue video messages denouncing you and forgiving the mob. And thi…

Is your country US? Somehow I think it is.

Oh yes, the United States, founded on religious freedom, is the place where you get stoned in the street for blasphemy.

Re: They stole my voice with AI

#242

Post by charlieyu1 »

Ygg2 wrote: post #227
kmlx wrote: post #162

Earlier quoted context omitted.

> what if AI was used to imitate a person saying something blasphemeous? > My country is already has blasphemy lynching mobs in your case the problem is not AI, it’s your country.

The problem is AI. What if you post video of a politician eating babies, and that causes some nutjob to kill that politician? Sure, distrust everything digital, but what if only evidence of someone doing something wrong is digital?

People copied and pasted faces of celebrities on another photo since 1990s. Nothing significant happened.

Re: They stole my voice with AI

#243

Post by cudgy »

criddell wrote: post #222
sureglymop wrote: post #171

Earlier quoted context omitted.

My specific fear is that if a picture of you next to your name is available online, that becomes part of the training set of a future model. Paranoically, I do not have any picture of myself available online. I could then trivially generate pictures or even videos of you e.g. by knowing your name. Of course that's just an example but I do think that's where we are headed and so the concept of "trust" will change a lo…

Do you have a state driver’s license? If so, then chances are data brokers have your photo from that. https://www.dallasnews.com/news/watchdog/2021/03/19/its-mind...

And your fingerprints …

Re: They stole my voice with AI

#244

Post by badgersnake »

yapyap wrote: post #206
ryzvonusef wrote: post #205

Earlier quoted context omitted.

But, but, it sounds so realistic ! Listen kiddo, I dunno what 'cryptographic signatures' are, all I know is it sounds exactly like movedx saying they likes pineapple on pizza, and I know what I heard, sounds just like them, heard them dozen of times on TV, must have been an undercover journalist who exposed them, I say a person who likes pineapple on pizza is not welcome in my house whatever you say, now be gone!

used the wrong example there, pineapple on pizza is a nice, delicious, refreshing topping

You are wrong.

Re: They stole my voice with AI

#245

Post by ryzvonusef »

pmarreck wrote: post #224
ryzvonusef wrote: post #135

Everyone has their own fears about AI, but my fears are especially chilling; what if AI was used to imitate a person saying something blasphemeous? My country is already has blasphemy lynching mobs based on the slightest perceived insult, real or imagined. They will mob you, lynch you, burn your corpse, then distribute sweets while you family hide and issue video messages denouncing you and forgiving the mob. And thi…

> My country is already has blasphemy lynching mobs based on the slightest perceived insult, real or imagined. They will mob you, lynch you, burn your corpse, then distribute sweets while you family hide and issue video messages denouncing you and forgiving the mob. Blasphemy laws—and the violence that sometimes accompanies them—are a cultural issue, not a technological one. When the risk of mob violence is in play,…

You are focusing too much on my specific problem instead of using it as a guide to understand your own situation.

Sure we have mobs and you don't, but we are talking about AI here.

Infact let's imagine a totally different culture to illustrate my point.

Imagine you are an Israeli, and people in your office have a habit of sending Whatsapp voice notes to confirm various things instead of calls, because that way you can have a record but don't have to type every damn thing out. Totally innocent and routine behaviour, you are just doing what many other people do.

A colleague pissed at you for whatever damn stupid reason creates a fake of your voice saying you support Hamas by using said voice notes, using an online tool that doesn't cost much or require much... are you saying just because you won't be lynched, that there isn't a problem?

You are confused why everyone is pissed at you and why suddenly your boss fired you, and by the time you find out the truth... the lie has spread to enough people in your social circle that there is no clearing your name.

Think of how little data in voice samples is required to generate an audio clip thats sounds very realistic, and how better it will get in an year. You don't need fancy PC or tech knowledge for that, already websites exist that do for cheap.

Just because you weren't lynched is no solace.

People are the problem, AI is just providing quality tools with minimal skill and cost required, thus broadening the user base.

Re: They stole my voice with AI

#246

Post by ryzvonusef »

charlieyu1 wrote: post #242
Ygg2 wrote: post #227

Earlier quoted context omitted.

The problem is AI. What if you post video of a politician eating babies, and that causes some nutjob to kill that politician? Sure, distrust everything digital, but what if only evidence of someone doing something wrong is digital?

People copied and pasted faces of celebrities on another photo since 1990s. Nothing significant happened.

problem is not celebrities, problem is general every day people.

With celeberaties, atleast they are famous enough that the idea of fakes exist... but you are Johnny nobody, that's not a fake, you just got caught and don't want to admit you said/did it!

The court of public opinion is much smaller and personal, and 'evidence' is much more realistic and detailed.

Re: They stole my voice with AI

#247

Post by GreenWatermelon »

latexr wrote: post #174
blueflow wrote: post #166

Earlier quoted context omitted.

> And we have no tools to control this. Do you know "The boy who cried wolf"? Fabricate some allegations yourself and this will train people to disbelieve them.

That sounds like a dangerous proposition. Either they fabricate allegations about a “nobody” and put them in trouble or they fabricate allegations about those in power and will be investigated and put themselves in danger. Neither strategy is good.

It doesn't have to be dangerous allegations. Fabricate a video of your country's dictator carrying a giant Boulder, with a flattering subtitle. This will send a message that videos could be deepfaked to such a degree.

Re: They stole my voice with AI

#248

Post by ahiknsr »

pmarreck wrote: post #223

[flagged]

> Scarlett's baseless accusation

https://www.theguardian.com/film/article/2024/jul/18/scarlet...

> The actor made headlines earlier this year when she issued a public statement saying OpenAI’s chief executive, Sam Altman, had contacted her in September 2023 to provide the voice for Sky, in an ode to her role in the Spike Jonze film Her, but she had declined for “personal reasons”. In May, as the company released Sky, Altman shared a tweet that simply read “her”.

Re: They stole my voice with AI

#249

Post by GreenWatermelon »

pjc50 wrote: post #154
vasco wrote: post #138

Earlier quoted context omitted.

The best we can hope for is that one personally avoids this for the first 5 years or so, and then it gets so widespread and easy that everyone will start doubting any videos they watch. The same way it took social media like reddit a few years of "finding the culprit" / "name and shame" till mods figured out that many times the online mob gets it wrong and so now that is usually not allowed. But many people will suff…

> everyone will start doubting any videos they watch. This kills the medium. Just as ubiquitous scam calls have moved people away from phones, this moves people away from using media which cannot be trusted. Done enough this destroys reporting and therefore democracy. I wonder when the first nonexistent candidate will be elected.

This sounds like saying Text, as a medium, is already destroyed. But as we can see, despite thousands of years of fraud potential, we still use text a medium. We're back to needing witnesses and corroborating evidence.

Re: They stole my voice with AI

#250

Post by CoastalCoder »

losvedir wrote: post #221
ryzvonusef wrote: post #135

Everyone has their own fears about AI, but my fears are especially chilling; what if AI was used to imitate a person saying something blasphemeous? My country is already has blasphemy lynching mobs based on the slightest perceived insult, real or imagined. They will mob you, lynch you, burn your corpse, then distribute sweets while you family hide and issue video messages denouncing you and forgiving the mob. And thi…

I think the answer, counterintuitively, is to make these AI tools more open and accessible. As long as they're restricted or regulated or inaccessible people will continue to think of videos and recordings as not fakeable. But make voice cloning something easy and fun to do with a $1 app, let the teens have their prank call fun and pretty soon it should work its way into the public consciousness. I had my 70 year mot…

> I had my 70 year mother ask me last week if she should remove her voicemail message because can't people steal her voice with it? I was surprised but I guess she heard it on a Fox segment or something.

Out of curiosity, how much training data is needed currently to mimic a voice at various levels of convincingness?

Post reply on HN