Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: Neural text to speech with dozens of celebrity voices

vocodes.com

Post reply on HN
Print view View original

31–40 of 174 posts

Re: Show HN: Neural text to speech with dozens of celebrity voices

#31

Post by minerjoe »

electriccello wrote: post #26
boarnoah wrote: post #17

On a more positive side to this technology. I've been wondering about the possibility of using this sort of tech (or the API offerings from Azure or GCP) to provide voice overs in video games. By that I mean for smaller budget Indie development, it would be certainly interesting to either be able to generate voice audio from transcripts in order to add voices to background NPCs and so on (or even the possibility of d…

Check out Modulate.ai! We make real-time, emotive voice skins aimed at gaming voice chat. Audio watermarking is also built-in to prevent fraud. Currently in a closed alpha stage but if you're part of a game studio and have interest please reach out!

Minor point. Its nice when people write URL's so that we can just follow as opposed to requiring mouse interactions to cut and paste.

Re: Show HN: Neural text to speech with dozens of celebrity voices

#32

Post by programmarchy »

echelon wrote: post #14
imjared wrote: post #12

Earlier quoted context omitted.

I've tried Gilbert Gottfried and NDT. I do get a console error about CORS: > Access to fetch at ' https://mumble.stream/speak_spectrogram' from origin ' https://vo.codes' has been blocked by CORS policy: No 'Access-Control-Allow-Origin' header is present on the requested resource. If an opaque response serves your needs, set the request's mode to 'no-cors' to fetch the resource with CORS disabled. Using Chrome stable

Oh man, I thought I had this CORS stuff sorted. Thanks for the help and info! I'm using version 84.0.4147.89 (Official Build) (64-bit) and getting back responses. I got the following response headers: access-control-allow-origin: https://vo.codes content-length: 151689 content-type: application/json date: Mon, 27 Jul 2020 15:55:37 GMT vary: Origin x-backend-hostname: tts-group-1-965d444f5-7kvkm I'll try to dump the c…

I'm also on Chrome (84) macOS, using the Craig Ferguson model.

I switched to Safari and Disabled CORS, but a 500 error is coming back now. So maybe the 500 response is the root cause, and the error handler is not returning CORS headers, masking the issue on Chrome.

Edit: by putting in a shorter input (sentence rather than paragraph) I was able to get a response.

Re: Show HN: Neural text to speech with dozens of celebrity voices

#33

Post by minerjoe »

echelon wrote: post #13
Firerouge wrote: post #3

I skimmed your about, where you mention it as a hobby demo of your deep work. Do you have a GitHub or technical documentation about how you build this sort of thing to work at scale?

I can make a blog post later, but at a high level: A rust TTS server hosts two models: a mel inference model and a mel inversion model. The ones I'm using are glow-tts and melgan. They fit together back to back in a pipeline. I chose these models not for their fidelity, but for their performance. They're 10x faster at inference than Tacotron 2. If you want something that sounds amazing, you're better off with a dense…

Can you share the cost of running this system?

Re: Show HN: Neural text to speech with dozens of celebrity voices

#34

Post by ethbro »

echelon wrote: post #28
ethbro wrote: post #25

Earlier quoted context omitted.

Out of curiosity, are there legal concerns? E.g. training off Schwarzenegger and offering an Arnold transform

I'm not a lawyer, but I think we're entering into a legal gray area. There are the existing frameworks of copyright, parody, free speech, slander, libel, etc. that are all somewhat tangential to this. I believe (I'm not certain) that celebrity voice impersonation is legal as long as it is not used to sell or endorse a product. Most models are trained on the original speaker's voice, but maybe only a little bit. Model…

I asked not because I expected an answer, but because I figured you'd have an insightful opinion.

It's essentially the performance of a composition vs the composition question again: at what point am I mimicking someone to the extent they have a valid claim on a portion of my work?

I expect it'll enter the courts a few milliseconds after someone clones a dead actor (without their estate's permission) for a new performance.

There's always been an inherent tension in the US distinction between a law of nature and a creative work though. It seems a bit silly for me to claim patent / trademark on a vector that encodes my likeness.

Re: Show HN: Neural text to speech with dozens of celebrity voices

#35

Post by kibwen »

GrantZvolsky wrote: post #27
searchableguy wrote: post #7

That's super cool. I am worried about the potential abuse of this service, are there any existing services that can help to identify audio deep fakes like this one is for making them? Found Resemblyzer: https://github.com/resemble-ai/Resemblyzer

Such a system will always suffer from false positives and false negatives. On a more positive note, when deepfakes become a problem, we will see the emergence of a culture where unsigned authoritative content is not paid any attention.

> we will see the emergence of a culture where unsigned authoritative content is not paid any attention

If current events are any indication, that culture will only emerge 30 years after the tech becomes widely usable, and in the interim will lead to absolute chaos in the form of weaponized disinformation.

Re: Show HN: Neural text to speech with dozens of celebrity voices

#36

Post by echelon »

programmarchy wrote: post #32
echelon wrote: post #14

Earlier quoted context omitted.

Oh man, I thought I had this CORS stuff sorted. Thanks for the help and info! I'm using version 84.0.4147.89 (Official Build) (64-bit) and getting back responses. I got the following response headers: access-control-allow-origin: https://vo.codes content-length: 151689 content-type: application/json date: Mon, 27 Jul 2020 15:55:37 GMT vary: Origin x-backend-hostname: tts-group-1-965d444f5-7kvkm I'll try to dump the c…

I'm also on Chrome (84) macOS, using the Craig Ferguson model. I switched to Safari and Disabled CORS, but a 500 error is coming back now. So maybe the 500 response is the root cause, and the error handler is not returning CORS headers, masking the issue on Chrome. Edit: by putting in a shorter input (sentence rather than paragraph) I was able to get a response.

I need better error messages, but I believe it should respond with something stating the length is too long.

What might've happened is that the instance your request was farmed out to might have been OOM killed. I've provided lots of memory, but these models are pretty massive and each inference run has to spin up a lot of matrices in memory.

This is all CPU inference, not GPU.

When the pods get OOM killed, they spin up again. The clusters for each speaker are about 5-10 pods apiece (with some double tenancy).

Re: Show HN: Neural text to speech with dozens of celebrity voices

#38

Post by echelon »

vedran wrote: post #37

This is great. I've been thinking about doing something similar with cartoon characters to build a Disney-style companion for my son as he gets older. I'm imagining something like an Alexa assistant but with Mickey Mouse's voice.

That's so cute! You should totally do it.

The hardest part of this is in dataset creation. It's hard to clean and annotate the data and can be quite manual. That's why companies with lots of data will win.

There are automated techniques to help with segmentation, bandpass filtering, transcriptions, etc., but they're far from perfect.

Re: Show HN: Neural text to speech with dozens of celebrity voices

#40

Post by mywittyname »

ethbro wrote: post #34
echelon wrote: post #28

Earlier quoted context omitted.

I'm not a lawyer, but I think we're entering into a legal gray area. There are the existing frameworks of copyright, parody, free speech, slander, libel, etc. that are all somewhat tangential to this. I believe (I'm not certain) that celebrity voice impersonation is legal as long as it is not used to sell or endorse a product. Most models are trained on the original speaker's voice, but maybe only a little bit. Model…

I asked not because I expected an answer, but because I figured you'd have an insightful opinion. It's essentially the performance of a composition vs the composition question again: at what point am I mimicking someone to the extent they have a valid claim on a portion of my work? I expect it'll enter the courts a few milliseconds after someone clones a dead actor (without their estate's permission) for a new perfor…

I suspect there's some plausible deniability built-in that might allow for such matters to be legal.

For example, lots of people sound like Arnold Schwarzenegger. So if you trained a model with tall, deep-voiced Austrian man, you could probably get something that people will immediately associate with Arnold without actually being his voice, or someone emulating him. Because much of what Americans associate with his voice is really a regional accent which is relatively uncommon in the US.

There may be a little bit more difficulty getting away with with someone like Gilbert Gottfried, whose voice is much more unique. But I do think you could get away with creating a voice that people think sounds just like him, but doesn't hold up in a side-by-side comparison.

What I think will happen is celebrities like Morgan Freeman will use their voice to train models like this, then gift these to their estates for use in the future.

Post reply on HN