Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: Neural text to speech with dozens of celebrity voices

vocodes.com

Post reply on HN
Print view View original

61–70 of 174 posts

Re: Show HN: Neural text to speech with dozens of celebrity voices

#61

Post by echelon »

maxerickson wrote: post #18

It doesn't like spelling mistakes. Ask it to say " The Aristoocrats!" with Gilbert Goddfried.

It's currently sourcing phonemes from a lookup table called CMUdict, which is constructed by Carnegie Mellon [1]. That database has 140,000 entries, but even so, you'd be surprised how many common words are omitted. And of course it is missing terms for things like "pokemon" and "fortnite", which I had to add myself.

I don't have generic grapheme -> phoneme/polyphone prediction, but that's something I look to add soon. In my literature review I didn't see anything in this space, so I was thinking I might have to come up with something novel.

[1] http://www.speech.cs.cmu.edu/cgi-bin/cmudict

Re: Show HN: Neural text to speech with dozens of celebrity voices

#63

Post by spdustin »

echelon wrote: post #13
Firerouge wrote: post #3

I skimmed your about, where you mention it as a hobby demo of your deep work. Do you have a GitHub or technical documentation about how you build this sort of thing to work at scale?

I can make a blog post later, but at a high level: A rust TTS server hosts two models: a mel inference model and a mel inversion model. The ones I'm using are glow-tts and melgan. They fit together back to back in a pipeline. I chose these models not for their fidelity, but for their performance. They're 10x faster at inference than Tacotron 2. If you want something that sounds amazing, you're better off with a dense…

> "…Instead of using graphemes, I'm using ARPABET phonemes…"

Is this why some examples I tried seemed to skip some of the words?

Re: Show HN: Neural text to speech with dozens of celebrity voices

#64

Post by pbhjpbhj »

mywittyname wrote: post #40
ethbro wrote: post #34

Earlier quoted context omitted.

I asked not because I expected an answer, but because I figured you'd have an insightful opinion. It's essentially the performance of a composition vs the composition question again: at what point am I mimicking someone to the extent they have a valid claim on a portion of my work? I expect it'll enter the courts a few milliseconds after someone clones a dead actor (without their estate's permission) for a new perfor…

I suspect there's some plausible deniability built-in that might allow for such matters to be legal. For example, lots of people sound like Arnold Schwarzenegger. So if you trained a model with tall, deep-voiced Austrian man, you could probably get something that people will immediately associate with Arnold without actually being his voice, or someone emulating him. Because much of what Americans associate with his…

> So if you trained a model with tall, deep-voiced Austrian man, you could probably get something that people will immediately associate with Arnold without actually being his voice, or someone emulating him. //

I think "passing off", an unregistered element of trademark laws, may be pertinent here. If the public think that there's an association and you're knowingly trading on that, even if the public are wrong, then you can be 'passing off' your output as someone else's goods/services/[vocal renditions].

It's likely you'd have to be very careful about use of copyright material for training the voice (eg extracting metrics that describe the voice). Fair Use might apply in USA though (even commercially).

IANAL, this is not legal advice.

Re: Show HN: Neural text to speech with dozens of celebrity voices

#65

Post by pbhjpbhj »

vedran wrote: post #37

This is great. I've been thinking about doing something similar with cartoon characters to build a Disney-style companion for my son as he gets older. I'm imagining something like an Alexa assistant but with Mickey Mouse's voice.

I know caselaw isn't settled at all on all this but I'd absolutely avoid posting anything on the web mentioning D' and the black and white mouse again unless you are interested in finding out firsthand how the law gets settled here ;o).

Not legal advice, of course.

Re: Show HN: Neural text to speech with dozens of celebrity voices

#66

Post by echelon »

spdustin wrote: post #63
echelon wrote: post #13

Earlier quoted context omitted.

I can make a blog post later, but at a high level: A rust TTS server hosts two models: a mel inference model and a mel inversion model. The ones I'm using are glow-tts and melgan. They fit together back to back in a pipeline. I chose these models not for their fidelity, but for their performance. They're 10x faster at inference than Tacotron 2. If you want something that sounds amazing, you're better off with a dense…

> "…Instead of using graphemes, I'm using ARPABET phonemes…" Is this why some examples I tried seemed to skip some of the words?

Exactly. If you type "I am a dangerous asdhfjahdsff velociraptor, rawr."

There aren't entries for

- asdhfjahdsff

- rawr

I added around 500 new words, but I missed a lot of stuff.

The ultimate fix is to have grapheme -> phoneme prediction so that all unseen words can be mapped to potential phonemes (polyphones).

Re: Show HN: Neural text to speech with dozens of celebrity voices

#67

Post by echelon »

minerjoe wrote: post #33
echelon wrote: post #13

Earlier quoted context omitted.

I can make a blog post later, but at a high level: A rust TTS server hosts two models: a mel inference model and a mel inversion model. The ones I'm using are glow-tts and melgan. They fit together back to back in a pipeline. I chose these models not for their fidelity, but for their performance. They're 10x faster at inference than Tacotron 2. If you want something that sounds amazing, you're better off with a dense…

Can you share the cost of running this system?

I can come back and post a write up. Please refresh this post later today.

I scaled for today, but it's pretty cheap to run day to day.

I also have some architectural optimizations to make that will greatly reduce the costs. Right now, nodes are responsible for two speakers apiece. This is an under-utilization since most speakers don't get used.

Re: Show HN: Neural text to speech with dozens of celebrity voices

#68

Post by echelon »

dsteinman wrote: post #57
mmastrac wrote: post #23

I'd love to have an option for Majel Barrett

I was going to mention the same. It would be a childhood dream come true to talk to my computer and have it talk back to me in the TNG computer voice.

That's a fantastic suggestion! I'll get to it!

Re: Show HN: Neural text to speech with dozens of celebrity voices

#69

Post by echelon »

minerjoe wrote: post #30

Hate to be that guy but I can't participate in this discussion due to javascript being required for the landing page. As an outlier not running javascript, I'm reaping what I sow, but it would be nice to me and others in the same boat if projects make their landing page viewable without the need for javascript.

You can POST to https://mumble.stream/speak for a raw waveform.

Here's a request:

curl 'https://mumble.stream/speak' --compressed -H 'Referer: https://vo.codes/' -H 'Content-Type: application/json' -H 'Origin: https://vo.codes' -H 'Connection: keep-alive' --data-raw '{"text":"testing 12345","speaker":"david-attenborough"}' --output output.wav

The other speaker values:

http://mumble.stream/speakers

Re: Show HN: Neural text to speech with dozens of celebrity voices

#70

Post by thiagocsf »

minerjoe wrote: post #30

Hate to be that guy but I can't participate in this discussion due to javascript being required for the landing page. As an outlier not running javascript, I'm reaping what I sow, but it would be nice to me and others in the same boat if projects make their landing page viewable without the need for javascript.

There’s a text area and a button to say what you typed.

Surely you can enable or use a browser with JavaScript when you choose to?

Post reply on HN