Show HN: Neural text to speech with dozens of celebrity voices • Hacker News Board

Re: Show HN: Neural text to speech with dozens of celebrity voices

#41

Post by mywittyname » Mon, Jul 27, 2020, 5:30 PM UTC

GrantZvolsky wrote: post #27

searchableguy wrote: post #7

That's super cool. I am worried about the potential abuse of this service, are there any existing services that can help to identify audio deep fakes like this one is for making them? Found Resemblyzer: https://github.com/resemble-ai/Resemblyzer

Such a system will always suffer from false positives and false negatives. On a more positive note, when deepfakes become a problem, we will see the emergence of a culture where unsigned authoritative content is not paid any attention.

This is a big issue!

Lots of bad things happen, and they are only surfaced because the person in question didn't notice the surreptitious recording. When deep fakes becomes a problem, it will give these people plausible deniability and they can just reject it as "fake news."

Re: Show HN: Neural text to speech with dozens of celebrity voices

#42

Post by kevinmchugh » Mon, Jul 27, 2020, 5:33 PM UTC

ethbro wrote: post #25

echelon wrote: post #24

Earlier quoted context omitted.

There are half a dozen startups in this space that provide the tech. They use embedded style tokens or sliders to change the emotion, pitch, timbre, etc. I don't have links off hand, but they're not too difficult to find. These companies tend to focus on off-the-shelf turnkey solutions, so they'll have a suite of a few voice actors to choose from for different character archetypes.

Out of curiosity, are there legal concerns? E.g. training off Schwarzenegger and offering an Arnold transform

It's not settled caselaw so anyone basing a business off this should expect to spend a lot of money defending it in courts

I saw once a company that offered to be the sole purveyor of a celebrity's synthesized voice. I haven't been able to find them again, but that seems like a much safer way to monetize this.

Re: Show HN: Neural text to speech with dozens of celebrity voices

#43

Post by mywittyname » Mon, Jul 27, 2020, 5:33 PM UTC

echelon wrote: post #11

mattigames wrote: post #9

I was wondering, wouldn't it be possible to classify the voice of every celebrity based on moods so one could make the voices less monotonic? So one could then add text metadata for the text-to-speech conversion, e.g. "[Angry] I have a dream, [Calm] but it has a patent so you can't copy it! (laughter) [Calm-fade-to-angry] In reality insomnia took it from me!"

Absolutely. These are called "style tokens" and they're an active area of TTS research. The problem is that currently your training data has to be annotated with these tokens, and that adds a lot to the difficulty of creating data sets. I imagine that over time this will get much easier to do.

Are there good emotion detectors for speech-to-text? Much like they have for facial recognition?

Re: Show HN: Neural text to speech with dozens of celebrity voices

#44

Post by echelon » Mon, Jul 27, 2020, 5:42 PM UTC

phantom_rehan wrote: post #39

Did you use machine learning ?

Yeah, forks of two open source pytorch models.

https://github.com/jaywalnut310/glow-tts

https://github.com/seungwonpark/melgan

Re: Show HN: Neural text to speech with dozens of celebrity voices

#45

Post by echelon » Mon, Jul 27, 2020, 5:50 PM UTC

aww_dang wrote: post #16

Needs more Christopher Walken

I both love your HN username (I hope you don't troll dang), and think that's an awesome suggestion. I don't know why it didn't occur to me.

Re: Show HN: Neural text to speech with dozens of celebrity voices

#46

Post by dralley » Mon, Jul 27, 2020, 6:17 PM UTC

boarnoah wrote: post #17

On a more positive side to this technology. I've been wondering about the possibility of using this sort of tech (or the API offerings from Azure or GCP) to provide voice overs in video games. By that I mean for smaller budget Indie development, it would be certainly interesting to either be able to generate voice audio from transcripts in order to add voices to background NPCs and so on (or even the possibility of d…

> On a more positive side to this technology.

I'm not sure that making it easier to profit off of the likeness of others is a positive side. If it's legal for indie studios to do, it's legal for 20th Century Fox, Universal, and so forth.

Re: Show HN: Neural text to speech with dozens of celebrity voices

#47

Post by sunsetMurk » Mon, Jul 27, 2020, 6:21 PM UTC

lowdose wrote: post #20

sunsetMurk wrote: post #15

Very cool, and easy to use! Can you give some more info on how you generated the models? I'm also interested in the tech stack you're using to implement this webapp... Would love some details! ..What's next?

> What's next? Text To Video webapp that renders text to video + voice synchronised of famous people. Who wouldn't like to laugh 5X more when social scrolling? The first platform that enables creators with the ability to produce deep fakes of celebs from text that they can broadcast as HQ video content to their audience will kill both Youtube & Instagram. Ranking based on likes so the best jokes of the day are trendi…

Awesome. here come the 'deepMemes'!

Re: Show HN: Neural text to speech with dozens of celebrity voices

#48

Post by blueblisters » Mon, Jul 27, 2020, 6:21 PM UTC

echelon wrote: post #22

galuggus wrote: post #19

How long does it take to generate a good quality voice?

I trained a base model on the Linda Johnson speech (LJS) data set for several days. I then transfer learned for each of these speakers. Some speakers have as little as 40 minutes of data, others have up to five hours. The resulting quality isn't strictly a function of the amount of training data, though more typically helps. It's also important to have high fidelity text transcriptions free of errors. The transfer le…

Have you had any success with using speaker embeddings to generate voices with fewer samples of speech? I did some cursory experiments but I couldn't get too far beyond getting pitch similar to the target speaker.

My reasoning for this approach: IMO, if the model learns a "universal human voice", it shouldn't need too much additional information to get a target voice.

Re: Show HN: Neural text to speech with dozens of celebrity voices

#49

Post by baxtr » Mon, Jul 27, 2020, 6:25 PM UTC

Please add Steve Jobs. I miss him

Re: Show HN: Neural text to speech with dozens of celebrity voices

#50

Post by dang » Mon, Jul 27, 2020, 6:49 PM UTC

echelon wrote: post #2

I've built a lot of celebrity text to speech models and host them online: https://vo.codes It has celebrities like Sir David Attenborough and Arnold Schwarzenegger, a bunch of the presidents, and also some engineers: PG, Sam Altman, Peter Thiel, Mark Zuckerberg I'm not far away from a working "real time" [1] voice conversion (VC) system. This turns a source voice into a target voice. The most difficult part is gettin…

(These comments originally were in https://news.ycombinator.com/item?id=23965106 but I've moved them)

We'll re-up that thread (see https://news.ycombinator.com/item?id=11662380 for how this works generally). I'm going to move this comment there as well because it includes more background info than you posted there.