Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

WaveNet: A Generative Model for Raw Audio

deepmind.com

Post reply on HN
Print view View original

41–50 of 153 posts

Re: WaveNet: A Generative Model for Raw Audio

#41

Post by CamperBob2 »

nhaliday wrote: post #28
swsieber wrote: post #21

Earlier quoted context omitted.

I think the point is that different parts of the story need different intonation patterns (reading a scary part vs a boring part, etc.). So in theory, it could be achieved by having multiple training sets (for the different intonation styles), along with analysis of the text to direct which part of the text needs what intonation. You might even be able to blend intonations.

Or just pay MTurk workers to annotate texts with intonation cues. I kinda doubt that would be profitable relative to just hiring readers, but in general you don't need to replace workers completely to cannibalize some of their wages/jobs.

Or treat it as part of the original author's job. When you write a piece of music you add tempo and intensity metadata to the score, so why not do the same when writing a novel?

Re: WaveNet: A Generative Model for Raw Audio

#42

Post by zardo »

infinite8s wrote: post #40
zardo wrote: post #18

Earlier quoted context omitted.

Basically the same idea as style transfer with image algorithms. Looking forward to Abraham Lincoln reading audiobooks to me.

That would require audio recordings of Abraham Lincoln's voice. Not sure recording technology existed back then.

Audio quality does leave something to be desired. https://vimeo.com/47987691

Re: WaveNet: A Generative Model for Raw Audio

#43

Post by kastnerkyle »

fastoptimizer wrote: post #5

Do they say how much time is the generation taking? Is this insanely slow to train but extremely fast to do generation?

Relatively, training is fast (due to parallelism / masking so you don't have to sample during training) but during generation sampling is a sequential process. They talk about it a bit in the previous papers for PixelCNN and PixelRNN.

Re: WaveNet: A Generative Model for Raw Audio

#47

Post by infinite8s »

zardo wrote: post #42
infinite8s wrote: post #40

Earlier quoted context omitted.

That would require audio recordings of Abraham Lincoln's voice. Not sure recording technology existed back then.

Audio quality does leave something to be desired. https://vimeo.com/47987691

Ah I stand corrected, thanks.

Re: WaveNet: A Generative Model for Raw Audio

#48

Post by novalis78 »

noonespecial wrote: post #11

So when I get the AI from one place, train it with the voices of hundreds of people from dozens of other sources, and then have it read a book from Project Gutenberg to an mp3... who owns the mechanical rights to that recording?

good point ... I am pretty sure there are a thousand audible products waiting to be launched.

Re: WaveNet: A Generative Model for Raw Audio

#49

Post by kastnerkyle »

dharma1 wrote: post #9

The samples sound amazing. These causal convolutions look like a great idea, will have to re-read a few times. All the previous generative audio from raw audio samples I've heard (using LSTM) has been super noisy. These are crystal clear. Dilated convolutions are already implemented in TF, look forward to someone implementing this paper and publishing the code.

I did a review for PixelCNN as a part of my summer internship, it covers a bit about how careful masking can be used to create a chain of conditional probabilities [0], which AFAIK is exactly how this "causal convolution" works (can't have dependencies in the 'future'). The PixelCNN and PixelRNN papers also cover this in a fair bit of detail. Ishaan Gulrajani's code is also a great implementation reference for PixelCNN / masking [1].

[0] https://github.com/tensorflow/magenta/blob/master/magenta/re...

[1] https://github.com/igul222/pixel_rnn/blob/master/pixel_rnn.p...

Re: WaveNet: A Generative Model for Raw Audio

#50

Post by novalis78 »

What's really intriguing is the part in their article where they explain the "babbling" of wavenet, when they train the network without the text input.

That sounds just like a small kid imitating a foreign (or their own) language. My kids grow up bilingual and I hear them attempt something similar when they are really small. I guess it's like listening in to their neural network modelling the sound of the new language.

Post reply on HN