Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

They stole my voice with AI

jeffgeerling.com

Post reply on HN
Print view View original

251–260 of 461 posts

Re: They stole my voice with AI

#251

Post by HeatrayEnjoyer »

losvedir wrote: post #221
ryzvonusef wrote: post #135

Everyone has their own fears about AI, but my fears are especially chilling; what if AI was used to imitate a person saying something blasphemeous? My country is already has blasphemy lynching mobs based on the slightest perceived insult, real or imagined. They will mob you, lynch you, burn your corpse, then distribute sweets while you family hide and issue video messages denouncing you and forgiving the mob. And thi…

I think the answer, counterintuitively, is to make these AI tools more open and accessible. As long as they're restricted or regulated or inaccessible people will continue to think of videos and recordings as not fakeable. But make voice cloning something easy and fun to do with a $1 app, let the teens have their prank call fun and pretty soon it should work its way into the public consciousness. I had my 70 year mot…

This is idealistic. People still haven't fully learned that images can be photoshopped in its twenty years of its existence. (Deep)faked porn is still harmful which is why it's a crime.

Worse, there isn't an attitude of default skepticism in many areas/cultures. If a person is suspected of violating the moral code the priority will be punishment and reinforcing that such behavior isn't acceptable. Whether or not the specific person actually did the specific act is a secondary concern.

It's just going to increase the number of people who will be harmed or killed.

Re: They stole my voice with AI

#252

Post by golol »

klabb3 wrote: post #231
golol wrote: post #115

Earlier quoted context omitted.

> AI is astonishingly good at laundering and remixing without exposing the training set They just about manage to make a good multimodal transformer that can generate audio and you expect that right away they can also interpolate in latent space? How does that actually work? It's not so simple. What benefit do they have from training on Scarlett Johansson's data, because they sure as hell have a big risk. They clearl…

To be clear I’m not saying it did happen for sure. Only that it’s realistic and not exactly far fetched given everything that’s publicly known. > they can also interpolate in latent space? How does that actually work? I don’t know. I assumed it is the case because of the vast and excellent interpolation AI appears to do in other domains.

> I don’t know. I assumed it is the case because of the vast and excellent interpolation AI appears to do in other domains.

Sure it's plausible just like many other forms of baseless speculation are, I still don't think to state it as a matter of fact like it is happening in this and other articles and in comments.

> I don’t know. I assumed it is the case because of the vast and excellent interpolation AI appears to do in other domains.

Yes but do you know of any examples with sequential data, and in particular transformers? I don't and I think it is a somewhat different thing. With pictures you just train an encoder into some low dimensional latent space and there you can interpolate, but working with sequences you don't want to give the whole sequence as an input to the model and generate the whole sequence as an output.

The natural way to do interpolation with an audio transformer model, which it can in principle indeed do, would be to somehow audio-prompt engineer it to speak in a voice that interpolates SJ and your voice actor. But that would be a pretty fragile method.

Re: They stole my voice with AI

#253

Post by djoldman »

> I remember when OpenAI practically cloned Scarlett Johanssen's voice ...

I don't have a dog in this fight but just to be clear, OpenAI has stated that they paid a voice actor to create the voice ("Sky") that sounds like Scarlett Johanssen. There was no "cloning" or "stealing" (that they say).

https://openai.com/index/how-the-voices-for-chatgpt-were-cho...

Re: They stole my voice with AI

#254

Post by input_sh »

bryanrasmussen wrote: post #181
underdeserver wrote: post #175

Earlier quoted context omitted.

Switzerland allows you to own a gun, but (generally) not to bear it. Huge difference.

America also has laws against taking guns into theaters and indiscriminately shooting people, once you have easy access to guns the gun becomes a potential solution to problems. Maybe Switzerland just has less problems where that potential solution appeals to people.

I'd say that the key difference is that Switzerland has compulsory military service in which you're taught how to operate a weapon properly. Therefore, everyone that has one has gone through months of training.

Vs the US, where there are loopholes that you can use to avoid even a basic background check, and then use it for the very first time to shoot someone.

Re: They stole my voice with AI

#255

Post by alternatex »

CoastalCoder wrote: post #250
losvedir wrote: post #221

Earlier quoted context omitted.

I think the answer, counterintuitively, is to make these AI tools more open and accessible. As long as they're restricted or regulated or inaccessible people will continue to think of videos and recordings as not fakeable. But make voice cloning something easy and fun to do with a $1 app, let the teens have their prank call fun and pretty soon it should work its way into the public consciousness. I had my 70 year mot…

> I had my 70 year mother ask me last week if she should remove her voicemail message because can't people steal her voice with it? I was surprised but I guess she heard it on a Fox segment or something. Out of curiosity, how much training data is needed currently to mimic a voice at various levels of convincingness?

Skype does it in real time (for live translation) with a few seconds of audio. For the reasons discussed in this thread, it continuously forgets its previous training from the call to not make the voice too similar, but just similar enough to distinguish the speakers.

Re: They stole my voice with AI

#256

Post by nkrisc »

ryzvonusef wrote: post #176
kmlx wrote: post #162

Earlier quoted context omitted.

> what if AI was used to imitate a person saying something blasphemeous? > My country is already has blasphemy lynching mobs in your case the problem is not AI, it’s your country.

Your country might not have lynching mobs, but you can't deny there are certain taboo topics in your society also, certain slurs and other opinions which would take you ages to clense and even then never fully. If an AI fake-porn of some ordinary person involving a minor was unleashed, think of the utter shame and horror they would be treated by people for the rest of their lives, even if it were proven false. No one…

Yes, you’re right. That’s why the problem is the lynch mobs. If there was no AI, people would find another way to sic the mob on someone. I’m sure it’s already happened countless times without AI. Mobs aren’t known for the rational behavior and respect of the rule of law.

Re: They stole my voice with AI

#257

Post by antegamisou »

XorNot wrote: post #32

The idea that stolen voice tones are going to matter at all is one of the shortest sighted bits of AI investment - powered by Hollywood "never make anything new" thinking. In about 5 years AI voices will be bespoke and more pleasant to listen to then any real human: they're not limited by vocal cord stress, can be altered at will, and can easily be calibrated by surveying user engagement. Subtly tweaking voice output…

Aesthetically disgusting take ew. Why is everyone like that here seriously.

Re: They stole my voice with AI

#258

Post by jerpint »

CoastalCoder wrote: post #250
losvedir wrote: post #221

Earlier quoted context omitted.

I think the answer, counterintuitively, is to make these AI tools more open and accessible. As long as they're restricted or regulated or inaccessible people will continue to think of videos and recordings as not fakeable. But make voice cloning something easy and fun to do with a $1 app, let the teens have their prank call fun and pretty soon it should work its way into the public consciousness. I had my 70 year mot…

> I had my 70 year mother ask me last week if she should remove her voicemail message because can't people steal her voice with it? I was surprised but I guess she heard it on a Fox segment or something. Out of curiosity, how much training data is needed currently to mimic a voice at various levels of convincingness?

Just a few seconds of speech gets you pretty far with a lot of models

Re: They stole my voice with AI

#259

Post by deepsun »

HeatrayEnjoyer wrote: post #251
losvedir wrote: post #221

Earlier quoted context omitted.

I think the answer, counterintuitively, is to make these AI tools more open and accessible. As long as they're restricted or regulated or inaccessible people will continue to think of videos and recordings as not fakeable. But make voice cloning something easy and fun to do with a $1 app, let the teens have their prank call fun and pretty soon it should work its way into the public consciousness. I had my 70 year mot…

This is idealistic. People still haven't fully learned that images can be photoshopped in its twenty years of its existence. (Deep)faked porn is still harmful which is why it's a crime. Worse, there isn't an attitude of default skepticism in many areas/cultures. If a person is suspected of violating the moral code the priority will be punishment and reinforcing that such behavior isn't acceptable. Whether or not the…

Yep, had a lawyer telling me that image timestamp cannot be faked. While it's literally a right-click away.

Re: They stole my voice with AI

#260

Post by exitb »

djoldman wrote: post #253

> I remember when OpenAI practically cloned Scarlett Johanssen's voice ... I don't have a dog in this fight but just to be clear, OpenAI has stated that they paid a voice actor to create the voice ("Sky") that sounds like Scarlett Johanssen. There was no "cloning" or "stealing" (that they say). https://openai.com/index/how-the-voices-for-chatgpt-were-cho...

This is subtly wrong. They tried to hire the celebrity, got refused, then hired a different talent to do her “natural voice”. The official story is that it just happens to sound alike.
Post reply on HN