It’s Game over on Vocal Deepfakes
daringfireball.net
It’s Game over on Vocal Deepfakes
1–10 of 122 posts
Re: It’s Game over on Vocal Deepfakes
#2I think he's right about this—things are going to get weird and ugly and people aren't prepared for what's coming. Here's my suggestion for safeguarding your sanity in the years ahead: find the people and the blogs you love, embrace RSS, block or avoid everything else, read old books, stop listening to podcasts, revert to email as the primary channel for occasional "social" correspondence, abandon all side projects that require additional screen time, go outside.
This is the way.
Re: It’s Game over on Vocal Deepfakes
#3So who's going to build the sex phone line where you can talk to celebrities like Marilyn Monroe?
Re: It’s Game over on Vocal Deepfakes
#4Once the generation times get to real-time, I dunno what's going to happen. I follow the voice acting community a lot and this is a big existential threat level of worry in that community.
But I also see some positives for the narrative voice field, but at the expense of actual actors.
The latest sequel to a favorite audio book series has the professional narrator pronouncing different character names and town names entirely different than the previous 7 books.
In another book the narrator completely changed the character voices in the sequel compared to the first.
The positives for listeners is eventually we can guarantee that voices are completely consistent between narrations. An editor will soon be able to describe the emotions of a character and nuance how the AI performs certain scenes.
It's really sad but I think the end of human audio storytelling is coming to and end quite rapidly.
Re: It’s Game over on Vocal Deepfakes
#5Those fake AI voices are perfect tools for phishing campaigns.
I wonder if ChatGPT combined with a voice generator can mount successful phishing campaigns.
It looks like Cyberpunk future we never asked is already here.
Re: It’s Game over on Vocal Deepfakes
#6I tried ElevenLabs and it is truly amazing. You don't need much audio to train it and the final result can be incredible. I shared some of the snippets with friends and relatives and after the intitial scare (same as John) we agreed that the outcome wouldn't be different than a few years ago...
We had impersonators for decades, so what have stopped political parties to hire an impersonator to create fake audios? The IA in this case is going to make it more accesible, but in political parties you don't want thousands of fake audios (they would lose credibility), you need only one.
Also, photoshopping photos would have a similar effect, and that technique has been available for years as well.
Re: It’s Game over on Vocal Deepfakes
#7> It’s all fun and games in these demos, but this is inevitably going to be put to use by ratfuckers to create fake scandals in political campaigns...And it feels inevitable that a Roger Stone or Steve Bannon type will use this technology to commission, say, a recording of Joe Biden forgetting his own name...
The road to hell is paved with good intentions, and all politicians are going to use this for their political campaigns, not just one specific side. Both.
Let's not veer off the wider point and believe that only one side will use it for bad things. All politicians are liars, and no matter what side they are on, if it benefits their agenda to influence the electorate to gain power, they will use it; even if it is used for spreading lies or false and misleading claims.
Re: It’s Game over on Vocal Deepfakes
#8Would a Roger Stone or Steve Bannon type fake a Russian collusion dossier and then run it through the media for literally years?
What are the "types" that would do something like this?
https://www.bbc.com/news/world-us-canada-59168626
https://www.theguardian.com/us-news/2022/oct/11/russian-anal...
https://www.nytimes.com/2022/10/18/us/politics/igor-danchenk...
https://www.npr.org/2021/11/12/1055030223/the-fbi-arrests-a-...
Re: It’s Game over on Vocal Deepfakes
#9Those fake AI voices are perfect tools for phishing campaigns. I wonder if ChatGPT combined with a voice generator can mount successful phishing campaigns. It looks like Cyberpunk future we never asked is already here.
[dead]
Re: It’s Game over on Vocal Deepfakes
#10So who's going to build the sex phone line where you can talk to celebrities like Marilyn Monroe?
For me, I don't see the point. But then I have a personality where that's not a product I would subscribe to or want. I have enough of a hard time connecting with real people. Having meaningless conversations as a paid service with a piece of software is gross to me.