Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: My wife might lose the ability to speak in 3 weeks – how to prepare?

news.ycombinator.com

Post reply on HN
Print view View original

141–150 of 229 posts

Re: Ask HN: My wife might lose the ability to speak in 3 weeks – how to prepare?

#141

Post by echelon »

I wrote https://trumped.com

You ideally want five hours of clean speech (good microphone, no background noise, high sample rate). It should be spoken clearly, in a single tone or mood. My model sounds awful because the data isn't consistent, and the room tone and microphones are terrible.

If you want different prosody or moods, don't mix them in the same data set.

You can experiment with transfer learning LJSpeech with Nvidia Tacotron2 right now. Glow-tts is also promising.

You'll start to get results with fifteen minutes of sample data, but for high quality you want a lot of audio.

Have your wife read a book and record it. The training chunks will be ~10 seconds apiece, so keep that in mind for how to segment the audio.

Focus on getting lots of good sounding data. Hours. The models will improve, but this may be your only shot of acquiring the data.

Download the LJSpeech dataset and listen to it. See how it sounds, how it's separated. That is a fantastic dataset that has yielded tremendous results, and you can use it for inspiration.

Re: Ask HN: My wife might lose the ability to speak in 3 weeks – how to prepare?

#142

Post by glonq »

Someone wrote: post #30
kerkeslager wrote: post #5

I don't have any answers to give you, but I want to say that this is a really loving and beautiful thing you're trying to do.

Is it? My first thought was “is your ideal also her ideal?” . We cannot rule out she wants to spend quality time with her partner instead of spending time in a recording studio, so that, if the worst outcome comes, her husband can remind her of what she lost.

That was also my first thought, having seen far too many tech geeks inflict unwanted products and projects onto their poor partners and families.

The sentiment is admirable, but it's a lot of work considering that the probability of a negative outcome is very low.

Re: Ask HN: My wife might lose the ability to speak in 3 weeks – how to prepare?

#143

Post by Deinos »

vervez wrote: post #87
audiohermit wrote: post #62

Earlier quoted context omitted.

I work in pathological speech processing/synthesis so I'm unfortunately familiar with your father's position. It really sucks that these people didn't know that archiving their voice would've been useful. I hear snippets that people manage to glean from family videos right after listening to their current voices and it makes me really sad. On the upside, your father can choose any celebrity he wants to voice him! Ton…

Is Morgan Freeman the most used celebrity?

A ranking of used voices would be fascinating. Especially broken down by user statistics.

Re: Ask HN: My wife might lose the ability to speak in 3 weeks – how to prepare?

#144

Post by bergerjac »

kemiller2002 wrote: post #129

My mom lost her ability to speak, and what you are going to find is that your life and how you interact with everyone will have to change. Human verbal communication is very fast. She will find it difficult to be part of normal conversations. Without lots of help, she will start to fade into the background of conversations, because she can't keep up. You will have to help her be a part of things. It will be a depress…

Seems like a great application for Elon's Neuralink.

Re: Ask HN: My wife might lose the ability to speak in 3 weeks – how to prepare?

#145

Post by joshribakoff »

reubenmorais wrote: post #83
joshribakoff wrote: post #57

Earlier quoted context omitted.

It's a corpus designed to capture the full breadth of combinatorial nuances of human speech in a general sense.

No, it is not. For one, it's a corpus of read speech, which means it does not capture well the characteristics of conversational human speech – hesitation, disfluencies, different tones and registers, etc. LibriSpeech has a paper explaining the design of the corpus, all you need to read is the first sentence of the abstract to know what it is supposed to capture: This paper introduces a new corpus of read English spe…

That sentence alone does not establish that read speech differs from conversational speech, thanks for the information / pointing this out, though.

Re: Ask HN: My wife might lose the ability to speak in 3 weeks – how to prepare?

#146

Post by Veedrac »

audiohermit wrote: post #38

Hey, speech ML researcher here. Make sure you have different recordings of different contexts. fifteen.ai's best TTS voices use ~90 min of utterances, some separated by emotion. If you're having her read a text, make sure it's engaging--we do a lot of unconscious voicing when reading aloud. Tbh, if she has a non-Anglophone accent, you're going to need more because the training data is biased towards UK/US speakers. I…

Also buy a (half) decent mic! They're much cheaper than you might expect.

Re: Ask HN: My wife might lose the ability to speak in 3 weeks – how to prepare?

#148

Post by sdenton4 »

audiohermit wrote: post #97
dheera wrote: post #88

Earlier quoted context omitted.

Which generator works the best, qualitatively? I come from a vision/ML background but haven't played with speech at all, so it's completely new to me, and wondering what the state of the art is. I've been wanting to create a TTS of myself so I can take phone calls using headphones and type back what I want to say so that I don't have to yell private information out loud in public locations. Would be nice if during no…

Much of the work in speech synthesis has been about closing the gap in vocoders, which take a generated spectrogram and output a waveform. There's a clear gap between practical online implementations and computational behemoths like WaveNet. As you implied it's hard to quantitatively judge which result is better, papers usually use surveys to judge. Here's a recent work that has a good comparison of some vocoders: ht…

WaveRNN (and even slimmer versions, like LPCNet) are great, and run for a tiny fraction of the compute of the original WaveNet. Pruning is also a good way to reduce model sizes.

I'm not sure what's up with the WaveGLOW (17.1M) example in the linked wavenode comparison... The base WaveGLOW sounds reasonable, though. They're also using all female voices, which strikes me as dodgy; lower male voice pitch tracking is often harder to get right, and a bunch of comparisons without getting into harder cases or failure modes makes it seem like they're covering something up.

(I've run into a bunch of comparisons for papers in the past where they clearly just did a bad job of implementing the prior art. There should be a special circle of hell...)

Re: Ask HN: My wife might lose the ability to speak in 3 weeks – how to prepare?

#149

Post by nighthawk454 »

This can be trained using only 5 Seconds of reference audio: https://google.github.io/tacotron/publications/speaker_adapt... https://arxiv.org/pdf/1806.04558.pdf

It's been mentioned a bit already, but thought it was worth calling out. This may be one of the lowest-overhead ways to start experimenting, at least in terms of data collection.

Re: Ask HN: My wife might lose the ability to speak in 3 weeks – how to prepare?

#150

Post by happycry »

louwhopley wrote: post #132
happycry wrote: post #14

We get quite a few requests for this at Resemble ( https://resemble.ai ). We can get her to record right on our website or you can upload an existing file (along with a video of her consent) on the platform. Feel free to shoot me a message and I'd be happy to help build a voice for her.

Wow, this looks like a great service! Out of interest what are the average response times to generate a clip of one or two sentences from a configured voice? Imagining the easy text-to-speech solution the OP could build on this resemble API.

Thanks! We do have a synchronous real-time API and latency is one of the bigger issues that we're trying to improve on now. At the moment, you can expect speeds that are 10x faster than realtime.
Post reply on HN