Ask HN: My wife might lose the ability to speak in 3 weeks – how to prepare? • Hacker News Board

Re: Ask HN: My wife might lose the ability to speak in 3 weeks – how to prepare?

#21

Post by jasonhn9999 » Thu, Jun 11, 2020, 4:07 PM UTC

When my dad lost his speech, we had Boogie Board Jot devices all over the house. It made writing short notes and simple dialogs much less tedious.

We also used the Verbally premium iPad app to help give him a voice and make transactions on easier.

Wishing you all the best.

Re: Ask HN: My wife might lose the ability to speak in 3 weeks – how to prepare?

#22

Post by trynewideas » Thu, Jun 11, 2020, 4:08 PM UTC

fxtentacle wrote: post #3

Record her reading the texts of a standardized text training corpus. That way, you can retrain an existing AI to do text to speech with her own voice. Edit: here's a link to the corpus that I believe Mozilla uses http://www.openslr.org/12/

Mozilla's is licensed CC-BY, which is pretty liberal. In case the Attribution license is a blocker, here's CMU_ARCTIC's, which is built from copyright-free sources and has no licensing restraints: http://festvox.org/cmu_arctic/

Re: Ask HN: My wife might lose the ability to speak in 3 weeks – how to prepare?

#23

Post by asveikau » Thu, Jun 11, 2020, 4:08 PM UTC

fxtentacle wrote: post #3

Record her reading the texts of a standardized text training corpus. That way, you can retrain an existing AI to do text to speech with her own voice. Edit: here's a link to the corpus that I believe Mozilla uses http://www.openslr.org/12/

Is she on board with this? I can imagine a lot of people being severely put off by being asked to record "a corpus of approximately 1000 hours" in advance of what sounds like a stressful surgery.

Re: Ask HN: My wife might lose the ability to speak in 3 weeks – how to prepare?

#24

Post by seesawtron » Thu, Jun 11, 2020, 4:09 PM UTC

Here's a recent work [0] where you can train the model with 10s audio and convert any "text to speech" (all doable in the browser). I tried with Google Colab demo [1] and its performance fluctuates with the training audio sample that you give it so might need some trial and error to get the sweet spot.

Also the model is not saved in the browser with Colab so you might also want to do it locally to save it eventualy (if it comes to that).

All the best mate!

[0] Main repo: https://github.com/CorentinJ/Real-Time-Voice-Cloning [1] Google colab repo to try it out: https://github.com/CorentinJ/Real-Time-Voice-Cloning/blob/ma...

Re: Ask HN: My wife might lose the ability to speak in 3 weeks – how to prepare?

#25

Post by fxtentacle » Thu, Jun 11, 2020, 4:09 PM UTC

The paper "Generalization Of Audio Deepfake Detection" gives an overview.

The paper https://arxiv.org/abs/1904.05441 has a list of spoofing methods.

Here's one method as paper https://arxiv.org/pdf/1806.04558.pdf

And here on GitHub https://github.com/CorentinJ/Real-Time-Voice-Cloning

Re: Ask HN: My wife might lose the ability to speak in 3 weeks – how to prepare?

#26

Post by zxter » Thu, Jun 11, 2020, 4:09 PM UTC

arslnjmn wrote: post #11

(off topic) Record a few things for her future self. E.g. favourite quotes, frequently used phrases.

Good advice! Maybe a few shoutouts to your future children.

Re: Ask HN: My wife might lose the ability to speak in 3 weeks – how to prepare?

#27

Post by ThePowerOfFuet » Thu, Jun 11, 2020, 4:10 PM UTC

david_draco wrote: post #6

Learn sign language, instead of holding on to the past.

> Learn sign language, instead of holding on to the past.

What the fuck, man?

Re: Ask HN: My wife might lose the ability to speak in 3 weeks – how to prepare?

#28

Post by josinalvo » Thu, Jun 11, 2020, 4:10 PM UTC

IDK about the tech, but I would not worry about it right now. You dont need to play with the tech unless the bad unlikely outcome comes to pass.

The only tip I have is from a bit of amateur sound editing I did: collect many samples, and beware of big phrases: Like, ask her to say the same thing many times. And ... sometimes ... to ... stop ... at ... each ... word. And ... so ... me ... ti ... mes at each syllable.

Otherwise, if you ever need to create a sample that contains a single word/syllable, you cant. It is weird how much sound that contains clearly distinguishable syllables for the human ears still is not separable when you go to edit it.

Also, you might want to check wordlists by frequency to get a menu of common words, and ipa notation, to ensure you cover a good range of sounds

Re: Ask HN: My wife might lose the ability to speak in 3 weeks – how to prepare?

#29

Post by thaumasiotes » Thu, Jun 11, 2020, 4:11 PM UTC

Some (decades old) research on this involved a research team creating a video of JFK saying "I never met Forrest Gump". I found a writeup in Google Books: https://books.google.com/books?id=mQtGVQeQplcC&pg=PA208&lpg=...

> We evaluated our Kennedy results qualitatively along the following dimensions: ... naturalness of the composited articulation; ...

Obviously the state of the art will have advanced, but maybe this can point the way toward more current research.

While I tend to agree with everyone else that this can be a great idea, my instinct is to float the idea to your wife first and see how she responds. I can imagine someone taking this negatively.

Re: Ask HN: My wife might lose the ability to speak in 3 weeks – how to prepare?

#30

Post by Someone » Thu, Jun 11, 2020, 4:12 PM UTC

kerkeslager wrote: post #5

I don't have any answers to give you, but I want to say that this is a really loving and beautiful thing you're trying to do.

Is it? My first thought was “is your ideal also her ideal?”.

We cannot rule out she wants to spend quality time with her partner instead of spending time in a recording studio, so that, if the worst outcome comes, her husband can remind her of what she lost.