Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

WaveNet: A Generative Model for Raw Audio

deepmind.com

Post reply on HN
Print view View original

31–40 of 153 posts

Re: WaveNet: A Generative Model for Raw Audio

#31

Post by augustl »

The music examples are utterly fascinating. It sounds insanely natural.

The only thing I can hear that sounds unnatural, is the way that the reverberation in the room (the "echo") immediately gets lower when the raw piano sound itself gets lower. In a real room, if you produce a loud sound and immediately after a soft sound, the reverberation of the loud sound remains. But since this network only models "the sound right now", the volume of the reverberation follows the volume of the piano sound.

To my ears, this is most prevalent in the last example, which starts out loud and gradually becomes softer. It sounds a bit like they are cross-fading between multiple recordings.

Regardless, the piano sounds completely natural to me, I don't hear any artifacts or sounds that a real piano wouldn't make. Amazing!

There are also fragments that sounds inspiring and very musical to my ears, such as the melody and chord progression after 00:08 in the first example.

Re: WaveNet: A Generative Model for Raw Audio

#33

Post by badminton1 »

espadrine wrote: post #15
grandalf wrote: post #7

This is incredible. I'd be worried if I were a professional audiobook reader :)

I wouldn't. The results they offer are excellent, but the missing points they need to achieve human level are related to producing the correct intonation, which requires accurate understanding of the material. That is still at least ten years in the future, I expect.

There is significant advance in sentiment analysis too. Trading bots use sentiment analysis as some of the input for their time series prediction algorithms. I would not say 10 years.

Re: WaveNet: A Generative Model for Raw Audio

#35

Post by dhammack »

erichocean wrote: post #25
dhammack wrote: post #23

Earlier quoted context omitted.

It seems like you're using WaveNet to do speech-to-text when we have better tools for that. To transfer text from Trump to Clinton, first run speech-to-text on Trump speech and then give that to a WaveNet trained on Clinton to generate speech that sounds like her but says the same thing as Trump.

> It seems like you're using WaveNet to do speech-to-text I'm proposing reducing a vocal performance into the corresponding WaveNet input. At no point in that process is the actual "text" recovered, and doing so would defeat the whole purpose, since I don't care about the text, I care about the performance of speaking the text (whatever it was). In your example, I can't force Trump to say something in particular. But…

I see. I still think it's easier to apply deepmind's feature transform on text rather than to try to invert a neural network. Armed with a network trained on Trump, deepmind's feature transform from text->network inputs, you should be able to make him say whatever you want, right?

Text -> features -> TrumpWaveNet -> Trump saying your text

Re: WaveNet: A Generative Model for Raw Audio

#36

Post by ThePhysicist »

augustl wrote: post #31

The music examples are utterly fascinating. It sounds insanely natural. The only thing I can hear that sounds unnatural, is the way that the reverberation in the room (the "echo") immediately gets lower when the raw piano sound itself gets lower. In a real room, if you produce a loud sound and immediately after a soft sound, the reverberation of the loud sound remains. But since this network only models "the sound ri…

I agree, the samples sound very natural. I ask myself though how similar they are to the data that has been used for training, as it would be trivial to rearrange individual pieces of a large training set in ways that sound good (especially if a human selects the good samples for presentation afterwards).

What I'd really like to see therefore is a systematic comparison of the generated music to the training set, ideally using a measure of similarity.

Re: WaveNet: A Generative Model for Raw Audio

#37

Post by erichocean »

dhammack wrote: post #35
erichocean wrote: post #25

Earlier quoted context omitted.

> It seems like you're using WaveNet to do speech-to-text I'm proposing reducing a vocal performance into the corresponding WaveNet input. At no point in that process is the actual "text" recovered, and doing so would defeat the whole purpose, since I don't care about the text, I care about the performance of speaking the text (whatever it was). In your example, I can't force Trump to say something in particular. But…

I see. I still think it's easier to apply deepmind's feature transform on text rather than to try to invert a neural network. Armed with a network trained on Trump, deepmind's feature transform from text->network inputs, you should be able to make him say whatever you want, right? Text -> features -> TrumpWaveNet -> Trump saying your text

> Armed with a network trained on Trump, deepmind's feature transform from text->network inputs, you should be able to make him say whatever you want, right?

Yes, that should work, and by tweaking the WaveNet input appropriately, you could also get him to say it in a particular way.

Re: WaveNet: A Generative Model for Raw Audio

#38

Post by augustl »

ThePhysicist wrote: post #36
augustl wrote: post #31

The music examples are utterly fascinating. It sounds insanely natural. The only thing I can hear that sounds unnatural, is the way that the reverberation in the room (the "echo") immediately gets lower when the raw piano sound itself gets lower. In a real room, if you produce a loud sound and immediately after a soft sound, the reverberation of the loud sound remains. But since this network only models "the sound ri…

I agree, the samples sound very natural. I ask myself though how similar they are to the data that has been used for training, as it would be trivial to rearrange individual pieces of a large training set in ways that sound good (especially if a human selects the good samples for presentation afterwards). What I'd really like to see therefore is a systematic comparison of the generated music to the training set, idea…

Good point! Are (some of) the chords completely made up, for example, or is it only using chords it has heard before?

Re: WaveNet: A Generative Model for Raw Audio

#39

Post by grav »

augustl wrote: post #38
ThePhysicist wrote: post #36

Earlier quoted context omitted.

I agree, the samples sound very natural. I ask myself though how similar they are to the data that has been used for training, as it would be trivial to rearrange individual pieces of a large training set in ways that sound good (especially if a human selects the good samples for presentation afterwards). What I'd really like to see therefore is a systematic comparison of the generated music to the training set, idea…

Good point! Are (some of) the chords completely made up, for example, or is it only using chords it has heard before?

Filtering out certain notes from a piano chord can be done by e.g. Melodyne, but that seems far from what's necessary to generate speech, so it would surprise me, if WaveNet can do that?

Re: WaveNet: A Generative Model for Raw Audio

#40

Post by infinite8s »

zardo wrote: post #18
erichocean wrote: post #14

This can be used to implement seamless voice performance transfer from one speaker to another: 1. Train a WaveNet with the source speaker. 2. Train a second WaveNet with the target speaker. Or for something totally new, train a WaveNet with a bunch of different speakers until you get one you like. This becomes the target WaveNet . 3. Record raw audio from the source speaker. Fun fact: any algorithmic process that "re…

Basically the same idea as style transfer with image algorithms. Looking forward to Abraham Lincoln reading audiobooks to me.

That would require audio recordings of Abraham Lincoln's voice. Not sure recording technology existed back then.
Post reply on HN