Live data from Hacker News

It’s Game over on Vocal Deepfakes

daringfireball.net

61–70 of 122 posts

Re: It’s Game over on Vocal Deepfakes

#61
post #32
post #4

Once the generation times get to real-time, I dunno what's going to happen. I follow the voice acting community a lot and this is a big existential threat level of worry in that community. But I also see some positives for the narrative voice field, but at the expense of actual actors. The latest sequel to a favorite audio book series has the professional narrator pronouncing different character names and town names…

My guess would be for voice actors to copyright their voices. I am not familiar with the voice generation ML but i guess for things like dramatic pauses a voice actor might still be needed. Also I'm guessing it should be fairly straightforward to detect such copyright violations. I would really like to see it implemented for heavy text driven rpgs. But overall yeah things look bad for voice actors in general

You already have the right to your likeness, which includes the things you say in your voice profile (especially when attached to your name, and not said in a manner like that of a parody or an impersonation for laughs).

The threat is that, supposedly, pretty much all of the media companies that need Voice actors have a clause that allows them to use generative voice AI after the VA's death or if they're otherwise incapacitated to the point where they would be physically unable to get the voice actor back in the studio to record more lines. This is not outlandish for voice actors since they probably couldn't care less about their voice being used after they are dead, but it does mean we'll be hearing a lot more of it as more and more decade-long franchises use it to avoid hiring a good-enough impersonation or voice replacement once the old one dies.

Re: It’s Game over on Vocal Deepfakes

#62

Those fake AI voices are perfect tools for phishing campaigns. I wonder if ChatGPT combined with a voice generator can mount successful phishing campaigns. It looks like Cyberpunk future we never asked is already here.

It's kind of amazing we're not seeing more of this already. It's a trivial application of the tech. We've already achieved a culture where the more batty something is the more likely people are to believe it, and now we have an infinite, personalized quackery generator. Without a massive cultural revolution putting the brakes on this, which I can't imagine coming from anywhere, I fear we're barreling into a future wh…

MacLuhan and others have already talked about this, from way back in the radio and TV generation.

I don't say this to mean, "oh, and they were wrong because everything turned out OK" If anything, we have had generations that grew up with this dysfunction in a normalized way, and it took another advanced in tech to really see it again.

(To take it further back into history, I think I now get what Cervantes was trying to say with the image of Don Quxiote tilting at windmills. It seemed so absurd, amusing, and quaint when I read it as a teen).

Re: It’s Game over on Vocal Deepfakes

#63

Does anyone have suggestions on verbal ways to authenticate family members, for example, on the phone?

One way: Everyone has a code-word which they only share with humans, over "private" channels like direct voice calls. For example, I'm "pineapple", and you're "orange", when we're connected on a call, we share our code-words in order to "authenticate" the call.

Another way: have a passphrase with a group of people that you share with each other, that is hard to guess but memorable. "The sky was green this morning" for example, that everyone in your family knows is the "human passphrase".

Another way: ask personal questions only the actual human being would know, like inside jokes or specific details about their life.

Another way: verify through some out-of-band method (like text message) in order to confirm their identity.

Another way: start everything with a video call where they have to pass their hand in front of their face before switching to audio only

None of the methods are perfect, all come with pro and cons.

But writing all of this made me feel really dystopian for some reason...

Re: It’s Game over on Vocal Deepfakes

#64

This is one of those things where Balaji is ahead of the curve - the way to guarantee metadata (e.g. who the speaker is) is to do it cryptographically on-chain. https://twitter.com/balajis/status/1583495595737481217

> Who via digital signature

the voice data signature can be removed, unless you mean something like an audio watermark

> What via hash

audio remuxing will destroy the hash, especially since it's already done by pretty much every social media company. Even with some sort of lossy audio hash, just put some clapping or cheering behind the voice and you've probably created a new audio hash.

> When via timestamp

file timestamps can change

This solves nothing unless your user either doesn't care about sharing the original file, or is malicious but too dumb to ffmpeg -i original.ogg out.mp3.

Re: It’s Game over on Vocal Deepfakes

#65
post #12

Earlier quoted context omitted.

Imagine an AI voice of Trump that phones lonely old people to thank them for coming to his rally. After 20 minutes of small talk he asks them to donate to his campaign via bitcoin...

Lonely old people sophisticated enough to do bitcoin? Color me skeptical.

They get conned into sending money from Bitcoin ATMs[0]

[0] https://www.atmmarketplace.com/news/elderly-michigan-couple-...

Re: It’s Game over on Vocal Deepfakes

#66
post #12

Earlier quoted context omitted.

Imagine an AI voice of Trump that phones lonely old people to thank them for coming to his rally. After 20 minutes of small talk he asks them to donate to his campaign via bitcoin...

Lonely old people sophisticated enough to do bitcoin? Color me skeptical.

I'd imagine an AI could quite patiently walk them through how to set it up and link their bank accounts. Honestly, it could be really helpful for tech support.

Re: It’s Game over on Vocal Deepfakes

#67
post #55

I was with him right up until the last paragraph. The pussy grabber tape was VIDEO. You can lip read it.

Most of the video, including the infamous "grab them by the pussy" quote, was a hot mic with Trump off-screen. So no, you can't lip read it.

Re: It’s Game over on Vocal Deepfakes

#70
post #47
post #36

Was anyone else not particularly impressed by this? I haven't listened to much Steve Jobs stuff but it sounded just stilted enough for me to think something was up. I'm not sure if it was because I came into it with a skeptical mindset because of the article and context around it. It also may be do to the fact that I have been using screen reading software for about 30 years. The only people who I've heard more then…

The audio directly in the post was for sure, but did you listen to the audio in the linked twitter thread? The only thing that gave me pause was the gpt text itself not the voice audio.

I had not listened to the audio in the twitter thread. It's a lot better and the only issue I notice is the pauses at the end of sentences don't seem quite right. I don't know if I'd notice the pauses though if I were not going into this with a suspicious mindset. After listening to the twitter thread I'm not convinced I could detect the difference if it was two random clips one generated and one not.
Post reply on HN