Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

It’s Game over on Vocal Deepfakes

daringfireball.net

Post reply on HN
Print view View original

21–30 of 122 posts

Re: It’s Game over on Vocal Deepfakes

#21

Post by th13row »

blueridge wrote: post #2

I think he's right about this—things are going to get weird and ugly and people aren't prepared for what's coming. Here's my suggestion for safeguarding your sanity in the years ahead: find the people and the blogs you love, embrace RSS, block or avoid everything else, read old books, stop listening to podcasts, revert to email as the primary channel for occasional "social" correspondence, abandon all side projects t…

No, this is not the way. What's the point in burying your head in the sand?

I don't think the vast majority of people 1) care about this 2) even know what RSS is 3) will stop listening to podcasts 4) will use email as their primary channel for correspondence.

By all means, do this if it's what you, personally, need.

Re: It’s Game over on Vocal Deepfakes

#22

Post by bobRollins »

geuis wrote: post #4

Once the generation times get to real-time, I dunno what's going to happen. I follow the voice acting community a lot and this is a big existential threat level of worry in that community. But I also see some positives for the narrative voice field, but at the expense of actual actors. The latest sequel to a favorite audio book series has the professional narrator pronouncing different character names and town names…

What will happen is humans redefine their role in society, and keep iterating literally on their survival.

We’ve been here before evolving what we call each others roles; priest, cleric, “people name Farmer must be farmers”; they’re arbitrary labels (in that “we don’t know why we exist”, not immutable obligations of reality way), and there’s no reason we to have see this as “the end of human storytelling”; software isn’t lobotomizing us, but changing who can reach us. If info distributed via technology cannot be trusted, good; past has shown there’s no reason to trust technology corporations anyway. Make it explicit.

Death of commercial figurative identity is not literal death and language is not the only source of understanding.

Re: It’s Game over on Vocal Deepfakes

#23

Post by svachalek »

DethNinja wrote: post #5

Those fake AI voices are perfect tools for phishing campaigns. I wonder if ChatGPT combined with a voice generator can mount successful phishing campaigns. It looks like Cyberpunk future we never asked is already here.

It's kind of amazing we're not seeing more of this already. It's a trivial application of the tech. We've already achieved a culture where the more batty something is the more likely people are to believe it, and now we have an infinite, personalized quackery generator.

Without a massive cultural revolution putting the brakes on this, which I can't imagine coming from anywhere, I fear we're barreling into a future where nothing is true anymore, a hall of mirrors where the capacity to fake anything at all far surpasses anyone's ability to make sense of it.

Re: It’s Game over on Vocal Deepfakes

#24

Post by DangitBobby »

101008 wrote: post #6

I tried ElevenLabs and it is truly amazing. You don't need much audio to train it and the final result can be incredible. I shared some of the snippets with friends and relatives and after the intitial scare (same as John) we agreed that the outcome wouldn't be different than a few years ago... We had impersonators for decades, so what have stopped political parties to hire an impersonator to create fake audios? The…

The difference is the scale, speed, and cost. You have to at least find an impersonator who can do a passable impersonation of the target, which may be a tall order. They can't impersonate everyone perfectly, and the really good ones are probably busy and/or expensive. Anyone can use this. Kids could use it.

Re: It’s Game over on Vocal Deepfakes

#25

Post by 2bitencryption »

On the flip side, if these deepfakes are so simple to generate, they will surely bombard us constantly throughout the day - radio, TV commercials, youtube videos, podcasts.

And people quickly become desensitized to that kind of thing. It could be the case that, after some initial "ramping up" period, these deepfakes are so cheap and abundant that no one falls for them.

When was the last time you got a call from a number you didn't recognize, and you actually picked it up? If you're like me, probably not in a long time, because you're aware it's almost certainly some scam/robo call.

Re: It’s Game over on Vocal Deepfakes

#26

Post by anaganisk »

DethNinja wrote: post #5

Those fake AI voices are perfect tools for phishing campaigns. I wonder if ChatGPT combined with a voice generator can mount successful phishing campaigns. It looks like Cyberpunk future we never asked is already here.

TikTok lady AI voice starts talking, hangs up phone

Re: It’s Game over on Vocal Deepfakes

#27

Post by lapitopi »

It is scary for sure. We were heading towards a brave new world even before chatGPT, where what's real was no longer to be taken for granted and you couldn't trust your eyes and ears. ChatGPT has just hastened things. I'm sure a new market will emerge for authentication services that verify the speaker or the person in the video similar to how we had twitter verified checkmark.

On the flip side I can't wait for someone to build a product where I record a few conversations with my parents while they are alive, and then when they're gone, through chatGPT + vocalfakes, I can have a parent forever. Sure, I'll know it's not the real thing, but when you really miss them, it certainly can make the pain a little less.

Re: It’s Game over on Vocal Deepfakes

#29

Post by mmcwilliams »

DethNinja wrote: post #5

Those fake AI voices are perfect tools for phishing campaigns. I wonder if ChatGPT combined with a voice generator can mount successful phishing campaigns. It looks like Cyberpunk future we never asked is already here.

Would it really be wise to run a criminal operation while piping all your requests through Microsoft servers, considering the history of the company collaborating with law enforcement?

Re: It’s Game over on Vocal Deepfakes

#30

Post by EGreg »

geuis wrote: post #4

Once the generation times get to real-time, I dunno what's going to happen. I follow the voice acting community a lot and this is a big existential threat level of worry in that community. But I also see some positives for the narrative voice field, but at the expense of actual actors. The latest sequel to a favorite audio book series has the professional narrator pronouncing different character names and town names…

Why only the end of human storytelling?

The bots will get all the capital as well as humans get laid off.

What will humans have to offer each other, let alone the corporations?

Next year they are coming out into the real world and replacing your manual labor… and they will never freak out and attack an old lady or lose their patience. And they will be perfectly tailored to everyone’s individual preferences and needs

https://www.axios.com/2023/03/17/robots-humanoid-figure-tesl...

Post reply on HN