Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

WaveNet implementation in Keras

github.com

Post reply on HN
Print view View original

11–17 of 17 posts

Re: WaveNet implementation in Keras

#11

Post by hiddencost »

robeastham wrote: post #7

I was very impressed with the TTS examples in the original DeepMind article ( https://deepmind.com/blog/wavenet-generative-model-raw-audio... ). Can someone elaborate on the usefulness of this implementation for Text-to-Speech? I'm keen to experiment with voice synthesis. I want to create dialog, from multiple voice sources, for some characters in a VR application that I'm working on. Perhaps this lib is a better opt…

Short answer: Don't use this for practical purposes. It takes 90 minutes to generate 1 second of audio.

Here's a good TTS system:

http://www.cstr.ed.ac.uk/projects/festival/

Re: WaveNet implementation in Keras

#12

Post by michael_h »

hiddencost wrote: post #11
robeastham wrote: post #7

I was very impressed with the TTS examples in the original DeepMind article ( https://deepmind.com/blog/wavenet-generative-model-raw-audio... ). Can someone elaborate on the usefulness of this implementation for Text-to-Speech? I'm keen to experiment with voice synthesis. I want to create dialog, from multiple voice sources, for some characters in a VR application that I'm working on. Perhaps this lib is a better opt…

Short answer: Don't use this for practical purposes. It takes 90 minutes to generate 1 second of audio. Here's a good TTS system: http://www.cstr.ed.ac.uk/projects/festival/

If you have phonetically rich source data, festival will work pretty well. If you need a little more flex in your system, and can deal with a super weird training process, HTS is probably a better choice. With a small amount of work, you can use consolidated HTS models from within festival. (http://hts.sp.nitech.ac.jp/)

Further, if you pine for the fjords of DNN-land, merlin (https://github.com/CSTR-Edinburgh/merlin) is brand new and looking to make things a little easier for everybody.

Re: WaveNet implementation in Keras

#13

Post by alphaBetaGamma »

Slightly off-topic question about WaveNet:

In the paper, they say that they double the dilation factor up to a limit and then repeat: 1, 2, 4, ..., 512, 1, 2, 4, ..., 512, 1, 2, 4, ..., 512

The doubling of the dilation factor makes sense to me, but what is happening with the "repeat" part? I don't understand what they are trying go do. Wouldn't make more sense to continue doubling?

Re: WaveNet implementation in Keras

#14

Post by basve »

alphaBetaGamma wrote: post #13

Slightly off-topic question about WaveNet: In the paper, they say that they double the dilation factor up to a limit and then repeat: 1, 2, 4, ..., 512, 1, 2, 4, ..., 512, 1, 2, 4, ..., 512 The doubling of the dilation factor makes sense to me, but what is happening with the "repeat" part? I don't understand what they are trying go do. Wouldn't make more sense to continue doubling?

My intuition is that the doubling up to 512 does increase the receptive field, but you're essentially building a non-linear convolutional filter with a kernel size of 1024. The network benefits from stacking multiple of these groups, because each group can again convolve over the previous outputs at every temporal distance, which allows for learning deeper/higher level features. It is similar to the stacked 2d convolutions used for images, where every subsequent convolutional layers learns more abstract and higher level features/attributes of the data. This is just intuition though, there is no evidence yet that this holds for wavenet's architecture.

Re: WaveNet implementation in Keras

#15

Post by robeastham »

hiddencost wrote: post #11
robeastham wrote: post #7

I was very impressed with the TTS examples in the original DeepMind article ( https://deepmind.com/blog/wavenet-generative-model-raw-audio... ). Can someone elaborate on the usefulness of this implementation for Text-to-Speech? I'm keen to experiment with voice synthesis. I want to create dialog, from multiple voice sources, for some characters in a VR application that I'm working on. Perhaps this lib is a better opt…

Short answer: Don't use this for practical purposes. It takes 90 minutes to generate 1 second of audio. Here's a good TTS system: http://www.cstr.ed.ac.uk/projects/festival/

But does anyone know if it's possible to do TTS with the recently released libraries?

Thanks for the links, but to my ear the samples on those links don't hit the mark. The Wavenet samples in the original article cross the threshold for me. So I'd like to try some short length dialog tests, especially as I've read elsewhere that 1 second only takes 4 minutes on a K80.

Any light anyone else can shed on this would be great.

Re: WaveNet implementation in Keras

#16

Post by basve »

robeastham wrote: post #15
hiddencost wrote: post #11

Earlier quoted context omitted.

Short answer: Don't use this for practical purposes. It takes 90 minutes to generate 1 second of audio. Here's a good TTS system: http://www.cstr.ed.ac.uk/projects/festival/

But does anyone know if it's possible to do TTS with the recently released libraries? Thanks for the links, but to my ear the samples on those links don't hit the mark. The Wavenet samples in the original article cross the threshold for me. So I'd like to try some short length dialog tests, especially as I've read elsewhere that 1 second only takes 4 minutes on a K80. Any light anyone else can shed on this would be g…

Afaik none of the released libraries support the TTS experiment described in the paper. Deepmind used pre-computed linguistic features to guide the system in generating natural sounding speech, so your output will probably depend on the quality of those features. For the sake of not spreading misinformation; the 4 minutes was measured using a small model with a sampling rate of 4khz, this would not generate something sounding like the samples from Deepmind.

Re: WaveNet implementation in Keras

#17

Post by robeastham »

basve wrote: post #16
robeastham wrote: post #15

Earlier quoted context omitted.

But does anyone know if it's possible to do TTS with the recently released libraries? Thanks for the links, but to my ear the samples on those links don't hit the mark. The Wavenet samples in the original article cross the threshold for me. So I'd like to try some short length dialog tests, especially as I've read elsewhere that 1 second only takes 4 minutes on a K80. Any light anyone else can shed on this would be g…

Afaik none of the released libraries support the TTS experiment described in the paper. Deepmind used pre-computed linguistic features to guide the system in generating natural sounding speech, so your output will probably depend on the quality of those features. For the sake of not spreading misinformation; the 4 minutes was measured using a small model with a sampling rate of 4khz, this would not generate something…

Thanks for the clarification and for spotting the 4khz error. This is fascinating stuff.

Looks like I'll have to concede that voice acting is much more practical, for now at least.

Post reply on HN