[deleted]
Ask HN: My wife might lose the ability to speak in 3 weeks – how to prepare?
31–40 of 229 posts
Re: Ask HN: My wife might lose the ability to speak in 3 weeks – how to prepare?
#32Learn sign language, instead of holding on to the past.
Ignoring how rude this comment is, I don't think embracing a technological solution could count as "holding on the the past".
Re: Ask HN: My wife might lose the ability to speak in 3 weeks – how to prepare?
#33Record her reading the texts of a standardized text training corpus. That way, you can retrain an existing AI to do text to speech with her own voice. Edit: here's a link to the corpus that I believe Mozilla uses http://www.openslr.org/12/
I think this is backwards... This is a corpus to train speech to text, not text to speech, right?
Re: Ask HN: My wife might lose the ability to speak in 3 weeks – how to prepare?
#34Record her reading the texts of a standardized text training corpus. That way, you can retrain an existing AI to do text to speech with her own voice. Edit: here's a link to the corpus that I believe Mozilla uses http://www.openslr.org/12/
Is she on board with this? I can imagine a lot of people being severely put off by being asked to record "a corpus of approximately 1000 hours" in advance of what sounds like a stressful surgery.
It's 1000 hours because multiple speakers record the same articles.
I believe some speakers only recorded 1-2 hours, which seems doable.
Re: Ask HN: My wife might lose the ability to speak in 3 weeks – how to prepare?
#35Re: Ask HN: My wife might lose the ability to speak in 3 weeks – how to prepare?
#36I don't have any answers to give you, but I want to say that this is a really loving and beautiful thing you're trying to do.
Depends. I'd be horrified to listen to my own voice when I am not speaking. Keep in mind, what we sound very different to ourselves.
Re: Ask HN: My wife might lose the ability to speak in 3 weeks – how to prepare?
#37I have had good results with this. https://www.resemble.ai/
It is based on this open source work. If you want to run it yourself. https://github.com/CorentinJ/Real-Time-Voice-Cloning
The voice cloning can be done in a matter of minutes. (Best of luck!
Re: Ask HN: My wife might lose the ability to speak in 3 weeks – how to prepare?
#38Hey, speech ML researcher here. Make sure you have different recordings of different contexts. fifteen.ai's best TTS voices use ~90 min of utterances, some separated by emotion. If you're having her read a text, make sure it's engaging--we do a lot of unconscious voicing when reading aloud. Tbh, if she has a non-Anglophone accent, you're going to need more because the training data is biased towards UK/US speakers.
If you want to read up on the basics, check out the SV2TTS paper: https://arxiv.org/pdf/1806.04558.pdf Basically you use a speaker encoding to condition the TTS output. This paper/idea is used all over, even for speech-to-speech translation, with small changes.
There's a few open-source version implementations but mostly outdated--the better ones are either private for business or privacy reasons.
There's a lot of work on non-parallel transfer learning (aka subjects are saying different things) so TTS has progressed rapidly and most public implementations lag a bit behind the research. If you're willing to grok speech processing, I'd start with NeMo for overall simplicity--don't get distracted by Kaldi.
Edit: Important note! Utterances are usually clipped of silence before/after so take that into account when analyzing corpus lengths. The quality of each utterance is much much more important than the length--fifteen.ai's TTS is so good primarily because they got fans of each character to collect the data.
Re: Ask HN: My wife might lose the ability to speak in 3 weeks – how to prepare?
#39get a great microphone and have her read her favorite books. Go for books with lots of dialog and emotional content.
Later, you can extract all the phonemes you want from it and you will retain the emotional expressiveness of her voice.
She should probably sing some songs -- lullabies, rock, etc. Go for emotional diversity.
Re: Ask HN: My wife might lose the ability to speak in 3 weeks – how to prepare?
#40Roger Ebert has some articles about his troubles he encountered that may be worth a read.