It’s Game over on Vocal Deepfakes • Hacker News Board

Re: It’s Game over on Vocal Deepfakes

#41

Post by rvz » Wed, Mar 22, 2023, 1:14 AM UTC

Teever wrote: post #13

rvz wrote: post #7

> It’s all fun and games in these demos, but this is inevitably going to be put to use by ratfuckers to create fake scandals in political campaigns...And it feels inevitable that a Roger Stone or Steve Bannon type will use this technology to commission, say, a recording of Joe Biden forgetting his own name... The road to hell is paved with good intentions, and all politicians are going to use this for their political…

Name the Roger Stone or Steve Bannon of the left.

Sure. The BLM fraudsters Alicia Garza, Patrisse Cullors, and Opal Tometi whose grifting campaigns brought in $180M for the organization only for them to spend it on themselves. More than what Roger Stone or Steve Bannon's worth combined.

Again. Let us not pretend that only one side can use this for their next so called 'donations' or 'fundraising campaign'. Both of them will, to influence the electorate of their choice back into power.

Re: It’s Game over on Vocal Deepfakes

#42

Post by [deleted] » Wed, Mar 22, 2023, 1:15 AM UTC

Teever wrote: post #13

rvz wrote: post #7

> It’s all fun and games in these demos, but this is inevitably going to be put to use by ratfuckers to create fake scandals in political campaigns...And it feels inevitable that a Roger Stone or Steve Bannon type will use this technology to commission, say, a recording of Joe Biden forgetting his own name... The road to hell is paved with good intentions, and all politicians are going to use this for their political…

Name the Roger Stone or Steve Bannon of the left.

[deleted]

Re: It’s Game over on Vocal Deepfakes

#43

Post by chatmasta » Wed, Mar 22, 2023, 1:15 AM UTC

13of40 wrote: post #18

DethNinja wrote: post #5

Those fake AI voices are perfect tools for phishing campaigns. I wonder if ChatGPT combined with a voice generator can mount successful phishing campaigns. It looks like Cyberpunk future we never asked is already here.

I'm sure it's going to get exponentially better, but right now ChatGPT sounds like one very vanilla guy. The other day I asked it to write me a conversation between David Foster Wallace and Ernest Hemingway, and it came out something like (paraphrasing): HEMINGWAY: I notice you write about political issues and philosophy. WALLACE: Thank you, I noticed you write about raw experience and manliness. HEMINGWAY: Well, wha…

I tried to get it to intentionally respond with grammatically incorrect sentences and it had a lot of trouble. I basically couldn't get it to work.

Re: It’s Game over on Vocal Deepfakes

#44

Post by ethicalsmacker » Wed, Mar 22, 2023, 1:15 AM UTC

geuis wrote: post #4

Once the generation times get to real-time, I dunno what's going to happen. I follow the voice acting community a lot and this is a big existential threat level of worry in that community. But I also see some positives for the narrative voice field, but at the expense of actual actors. The latest sequel to a favorite audio book series has the professional narrator pronouncing different character names and town names…

On the upside, I can now narrate my own life and thoughts as Morgan Freeman.

Re: It’s Game over on Vocal Deepfakes

#45

Post by NoZebra120vClip » Wed, Mar 22, 2023, 1:15 AM UTC

rdedev wrote: post #32

geuis wrote: post #4

Once the generation times get to real-time, I dunno what's going to happen. I follow the voice acting community a lot and this is a big existential threat level of worry in that community. But I also see some positives for the narrative voice field, but at the expense of actual actors. The latest sequel to a favorite audio book series has the professional narrator pronouncing different character names and town names…

My guess would be for voice actors to copyright their voices. I am not familiar with the voice generation ML but i guess for things like dramatic pauses a voice actor might still be needed. Also I'm guessing it should be fairly straightforward to detect such copyright violations. I would really like to see it implemented for heavy text driven rpgs. But overall yeah things look bad for voice actors in general

Conceivably, anyone who uses his or her voice for spoken word or singing could, in theory, cryptographically sign each master copy of the recording, and therefore certify that they were present and active in its creation. Of course any post-processing would lose this attestation. So it's practically useless, I suppose. But it would be a first-line of defense in copyright cases, at least, by being able to say "I did indeed say that, and here's the signature." or "You can't prove that I said and signed that."

Re: It’s Game over on Vocal Deepfakes

#46

Post by ekianjo » Wed, Mar 22, 2023, 1:15 AM UTC

rdedev wrote: post #32

geuis wrote: post #4

Once the generation times get to real-time, I dunno what's going to happen. I follow the voice acting community a lot and this is a big existential threat level of worry in that community. But I also see some positives for the narrative voice field, but at the expense of actual actors. The latest sequel to a favorite audio book series has the professional narrator pronouncing different character names and town names…

My guess would be for voice actors to copyright their voices. I am not familiar with the voice generation ML but i guess for things like dramatic pauses a voice actor might still be needed. Also I'm guessing it should be fairly straightforward to detect such copyright violations. I would really like to see it implemented for heavy text driven rpgs. But overall yeah things look bad for voice actors in general

Trademark of the voices, not copyright...

Re: It’s Game over on Vocal Deepfakes

#47

Post by uberman » Wed, Mar 22, 2023, 1:15 AM UTC

jareds wrote: post #36

Was anyone else not particularly impressed by this? I haven't listened to much Steve Jobs stuff but it sounded just stilted enough for me to think something was up. I'm not sure if it was because I came into it with a skeptical mindset because of the article and context around it. It also may be do to the fact that I have been using screen reading software for about 30 years. The only people who I've heard more then…

The audio directly in the post was for sure, but did you listen to the audio in the linked twitter thread? The only thing that gave me pause was the gpt text itself not the voice audio.

Re: It’s Game over on Vocal Deepfakes

#48

Post by kevin_thibedeau » Wed, Mar 22, 2023, 1:16 AM UTC

DethNinja wrote: post #5

Those fake AI voices are perfect tools for phishing campaigns. I wonder if ChatGPT combined with a voice generator can mount successful phishing campaigns. It looks like Cyberpunk future we never asked is already here.

We can just make the AI phishing bots talk to a Lenny 2.0 trained to tie them up in endless conversations. AI interacting with AI will be the next economic boom fomenting a wave of new wealth creation.

Re: It’s Game over on Vocal Deepfakes

#49

Post by ethicalsmacker » Wed, Mar 22, 2023, 1:16 AM UTC

DethNinja wrote: post #5

Those fake AI voices are perfect tools for phishing campaigns. I wonder if ChatGPT combined with a voice generator can mount successful phishing campaigns. It looks like Cyberpunk future we never asked is already here.

"Hey mom, it's X. Sorry to leave this voice mail, I got in a bit of a car accident and lost my phone. I need to get back home, could you send $$$$ to ####?"

Re: It’s Game over on Vocal Deepfakes

#50

Post by stubish » Wed, Mar 22, 2023, 1:16 AM UTC

DethNinja wrote: post #5

Those fake AI voices are perfect tools for phishing campaigns. I wonder if ChatGPT combined with a voice generator can mount successful phishing campaigns. It looks like Cyberpunk future we never asked is already here.

You can already fool real world voice authentication systems. No need to phish for credentials when the voice is the credential: https://www.theguardian.com/technology/2023/mar/16/voice-sys...