Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

They stole my voice with AI

jeffgeerling.com

Post reply on HN
Print view View original

321–330 of 461 posts

Re: They stole my voice with AI

#321

Post by eth0up »

meowster wrote: post #311
eth0up wrote: post #225

My aunt is (supposedly) in her 90s, having left the US for Ecuador 10 or so years ago. We've remained in regular contact via phone over the years. Recently we went more than a few months without speaking. She has/had two numbers; magic jack and google. When I tried to call her, the magic jack was no longer in service and google said something about "unavailable". I reached out to my cousin (my aunt's daughter) to inq…

It's possible if she doesn't know why you're asking the questions and requesting a specific photograph, her responses won't be helpful. For example, if you asked for a selfie, she might just think you want a picture of her, and she remembers that she has a picture of her that she took last year where she looked good (passport phot that people dress up for), and wants you to have a good photo rather than one where she…

Hey there,

I couldn't include our entire dialogue into an HN comment, but yes, upon prodding as deeply as I could and running out of ideas, I explained my suspicions. The response wasn't what I expected, but not direct evidence supporting my concers quite either.

If it was my aunt, she understands well. If not, the perpetrator does too.

One of a few other instances which got my attention was a voicemail she left, which I retain a recording of. It starts by saying her name, awkwardly, followed by a 5-8 second pause, then saying "Hi. This is . I always refer to her by her abbreviated single syllable name, while the voicemail used her formal, full name.

I haven't heard from her since saying that if anything went wrong, I'd be looking for fingerprints on the passport.

Re: They stole my voice with AI

#322

Post by HeatrayEnjoyer »

benterix wrote: post #197
ryzvonusef wrote: post #135

Everyone has their own fears about AI, but my fears are especially chilling; what if AI was used to imitate a person saying something blasphemeous? My country is already has blasphemy lynching mobs based on the slightest perceived insult, real or imagined. They will mob you, lynch you, burn your corpse, then distribute sweets while you family hide and issue video messages denouncing you and forgiving the mob. And thi…

I'm very sorry to say this but if you live in a country that is killing others for what they say, AI is probably not your biggest problem. And I don't believe an easy solution exists.

"You can't refuse this brand new technology but you must change your society's culture that's been around for centuries so you are compatible with it." is a repulsively Silicon Valley answer.

Re: They stole my voice with AI

#323

Post by berniedurfee »

kmlx wrote: post #162
ryzvonusef wrote: post #135

Everyone has their own fears about AI, but my fears are especially chilling; what if AI was used to imitate a person saying something blasphemeous? My country is already has blasphemy lynching mobs based on the slightest perceived insult, real or imagined. They will mob you, lynch you, burn your corpse, then distribute sweets while you family hide and issue video messages denouncing you and forgiving the mob. And thi…

> what if AI was used to imitate a person saying something blasphemeous? > My country is already has blasphemy lynching mobs in your case the problem is not AI, it’s your country.

What country is immune to this?

As far as I can tell the collective conscious of every country is swayed by propaganda.

A written headline is enough to incite rage in any country much less a voice or video indistinguishable from the real thing.

Folks in “developed’ countries have their lives destroyed or ended all the time based on rumors of something said or done.

Re: They stole my voice with AI

#324

Post by HeatrayEnjoyer »

CoastalCoder wrote: post #250
losvedir wrote: post #221

Earlier quoted context omitted.

I think the answer, counterintuitively, is to make these AI tools more open and accessible. As long as they're restricted or regulated or inaccessible people will continue to think of videos and recordings as not fakeable. But make voice cloning something easy and fun to do with a $1 app, let the teens have their prank call fun and pretty soon it should work its way into the public consciousness. I had my 70 year mot…

> I had my 70 year mother ask me last week if she should remove her voicemail message because can't people steal her voice with it? I was surprised but I guess she heard it on a Fox segment or something. Out of curiosity, how much training data is needed currently to mimic a voice at various levels of convincingness?

Almost none, at most as little as a professional impersonator requires. GPT-4o's advanced voice mode would clone the user's voice by accident. A recording clip of one incident is available:

>During testing, we also observed rare instances where the model would unintentionally generate an output emulating the user’s voice.

>Example of unintentional voice generation, model outbursts “No!” then begins continuing the sentence in a similar sounding voice to the red teamer’s voice

https://assets.ctfassets.net/kftzwdyauwt9/4CG0G7y9WOfEkzBpi7...

https://openai.com/index/gpt-4o-system-card/

Re: They stole my voice with AI

#325

Post by nuancebydefault »

ryzvonusef wrote: post #272
nkrisc wrote: post #256

Earlier quoted context omitted.

Yes, you’re right. That’s why the problem is the lynch mobs. If there was no AI, people would find another way to sic the mob on someone. I’m sure it’s already happened countless times without AI. Mobs aren’t known for the rational behavior and respect of the rule of law.

I was simply explaining my personal fears regarding AI given that lynch mobs exist in my country; the lesson you should have taken from that was not about lynch mobs, but to analyse how AI might affect YOU in your particular environment. Remember the post is about cloning a youtuber's voice using AI; most people were thinking of the copywrite aspect of it, but I wanted to share with people what my fear was. My fears…

Maybe it's because I have no knowledge of lynch mobs in my country but I find that fear is a bad adviser. I think currently there is no way back in AI development. I read Microsoft is getting a failed nuclear reactor repaired to power the power hungry AI.

The only way forward is making sure most people know that digital content can be easily faked. If you hand out good faker tools for free, it will happen faster

Re: They stole my voice with AI

#326

Post by whamlastxmas »

shevekofurras wrote: post #213
godelski wrote: post #140

Earlier quoted context omitted.

> what if AI was used to imitate a person saying something blasphemeous? I've been contemplating writing an open letter to Dang to nuke my account. Because at this time you can likely deanonymize any user with a fair amount of comments. As long as you can correlate. You can certainly steal their language, even if not 100% accurate. It may be caution, but it isn't certain that we won't enter a dark forest and there's…

You can't nuke your account. You can close it but your comments remain on the site. They'll delete your account and assign your comments and posts to a random username. Yes this violates any EU citizen's right to be forgotten under GDPR. Welcome to silicon valley.

Does every comment previously attached to your account have a unique username or do they still all share the same?

Sort of moot point considering the multiple HN archives that would still have the original username attached

Re: They stole my voice with AI

#327

Post by berniedurfee »

toomuchtodo wrote: post #11
wwweston wrote: post #4

I appreciate his pointer to precedent, but the truth is that while precedent is a start, we're going to need to do work with principles beyond precedent. When tech introduces unprecedented capabilities, we will either figure out how to draw boundaries within which it (among other features of society) works for people, not against them, or we'll let it lead us closer to a world in which the strong do what they will an…

California recently signed some legislation into effect. It’s a start. Congress is working on “No Artificial Intelligence Fake Replicas And Unauthorized Duplications Act.” Still in dev in the House, but has bipartisan support. Call your congressperson, ask them to co-sponsor and/or vote for it. https://www.cbsnews.com/losangeles/news/california-bills-pro... https://salazar.house.gov/media/press-releases/salazar-intro…

No doubt it’s bipartisan!

Politician’s careers live and die in the fickle Court of Public Opinion. They’re probably the most susceptible cohort to AI fakes.

One of the rare times, it seems, that politician’s incentives are aligned with the populous. (Yes, I could have left that last part out.)

Re: They stole my voice with AI

#328

Post by memothon »

eth0up wrote: post #225

My aunt is (supposedly) in her 90s, having left the US for Ecuador 10 or so years ago. We've remained in regular contact via phone over the years. Recently we went more than a few months without speaking. She has/had two numbers; magic jack and google. When I tried to call her, the magic jack was no longer in service and google said something about "unavailable". I reached out to my cousin (my aunt's daughter) to inq…

I'm imagining your poor 90 year old aunt playing this wild game of Simon says with you and having no idea what's going on.

Maybe just ask the cousin not to send any more money?

Re: They stole my voice with AI

#329

Post by rustcleaner »

pjc50 wrote: post #193
kmlx wrote: post #162

Earlier quoted context omitted.

> what if AI was used to imitate a person saying something blasphemeous? > My country is already has blasphemy lynching mobs in your case the problem is not AI, it’s your country.

The US equivalent is much less labour intensive than a lynch mob: it's mass shooters radicalized by things they've read on the internet. Or https://www.npr.org/2024/09/19/nx-s1-5114047/springfield-ohi... , where repeating racial libel causes a public safety problem. While this kind of incitement in no way requires AI, it's certainly something that's easier to do when you can fake evidence. See also https://www.bbc.co…

>radicalized by things they've read on the internet

That take pretty much throws the shooters' preceding years of anguish under the bus to make a political score. "Ban the wrongspeech!" is a seductively easier proposition than ending youth bullying and ostracism.

Re: They stole my voice with AI

#330

Post by geerlingguy »

rishikeshs wrote: post #187
geerlingguy wrote: post #186

Earlier quoted context omitted.

It's part of the coat of arms for the Geerling family, yes.

Thanks. Would be great if you could write a bit about it.

Honestly I'm not the heraldry guy or ancestry guy in my family, but I thought it looked cool and helped make an SVG of it for a shirt my uncle made for all the family one year, so I slapped it in the footer of the blog :)
Post reply on HN