Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

WaveNet: A Generative Model for Raw Audio

deepmind.com

Post reply on HN
Print view View original

81–90 of 153 posts

Re: WaveNet: A Generative Model for Raw Audio

#81

Post by Scaevolus »

TheOtherHobbes wrote: post #77
augustl wrote: post #31

The music examples are utterly fascinating. It sounds insanely natural. The only thing I can hear that sounds unnatural, is the way that the reverberation in the room (the "echo") immediately gets lower when the raw piano sound itself gets lower. In a real room, if you produce a loud sound and immediately after a soft sound, the reverberation of the loud sound remains. But since this network only models "the sound ri…

I can hear some distortion in the piano notes - which may be an audio compression artefact, or it may be the output of the resynthesis process. If you train NNs at the phrase level and overfit, then you get something that is indeed more or less the same as cross-fading at random between short sections. Piano music is very idiomatic, so you'll capture some typical piano gestures that way. But I'd be surprised if the m…

The samples are uncompressed WAV files, so everything you hear is a direct result of the synthesis process. Some of the distortion is a result of the 16kHz sample rate-- it's not 44.1kHz CD quality.

Re: WaveNet: A Generative Model for Raw Audio

#82

Post by syllogism »

espadrine wrote: post #15
grandalf wrote: post #7

This is incredible. I'd be worried if I were a professional audiobook reader :)

I wouldn't. The results they offer are excellent, but the missing points they need to achieve human level are related to producing the correct intonation, which requires accurate understanding of the material. That is still at least ten years in the future, I expect.

Not really. They're training directly on the waveform, so the model can learn intonation. They just need to train on longer samples, and perhaps augment their linguistic representation with some extra discourse analysis.

A big problem with generating prosody has always been that our theories of it don't really provide a great prediction of people's behaviours. It's also very expensive to get people to do the prosody annotations accurately, using whatever given theory.

Predicting the raw audio directly cuts out this problem. The "theory" of prosody can be left latent, rather than specified explicitly.

Re: WaveNet: A Generative Model for Raw Audio

#83

Post by space_fountain »

novalis78 wrote: post #79
sjwright wrote: post #76

Earlier quoted context omitted.

To my Australian English ears, the babbling sounded vaguely Scandinavian.

Indeed. I was surprised by that as well. Sounded like a Dutch speaker with a muffled voice behind a screen.

Might just be that English is fairly close to German and the like but as English speakers it doesn't sound like English to us because we know English so it gets mapped as a similar but different language.

Re: WaveNet: A Generative Model for Raw Audio

#84

Post by Applejinx »

chestervonwinch wrote: post #29

Is it possible to use the "deep dream" methods with a network trained for audio such as this? I wonder what that would sound like, e.g., beginning with a speech signal and enhancing with a network trained for music or vice versa.

The piano stuff already seemed like 'dream music', as did the 'babble' examples. I found myself terribly frustrated by how short all those examples were. I wanted lots more :)

Re: WaveNet: A Generative Model for Raw Audio

#85

Post by mrkgnao »

DarkTree wrote: post #60
augustl wrote: post #31

The music examples are utterly fascinating. It sounds insanely natural. The only thing I can hear that sounds unnatural, is the way that the reverberation in the room (the "echo") immediately gets lower when the raw piano sound itself gets lower. In a real room, if you produce a loud sound and immediately after a soft sound, the reverberation of the loud sound remains. But since this network only models "the sound ri…

It shot me forward to a time where people just click a button to generate music they want to listen to. If you really like the generation, you save it and share it. It wouldn't have all of the other aspects that we derive from human-produced music like soul/emotion (because we know it's coming from a human, not because of how it sounds), but it would be a cool application idea anyway.

This reminds me of the Library of Babel short story.

Re: WaveNet: A Generative Model for Raw Audio

#87

Post by ycombinatorMan »

syllogism wrote: post #82
espadrine wrote: post #15

Earlier quoted context omitted.

I wouldn't. The results they offer are excellent, but the missing points they need to achieve human level are related to producing the correct intonation, which requires accurate understanding of the material. That is still at least ten years in the future, I expect.

Not really. They're training directly on the waveform, so the model can learn intonation. They just need to train on longer samples, and perhaps augment their linguistic representation with some extra discourse analysis. A big problem with generating prosody has always been that our theories of it don't really provide a great prediction of people's behaviours. It's also very expensive to get people to do the prosody…

theres 0 chance of effective intonation and tone without understanding of the material

Re: WaveNet: A Generative Model for Raw Audio

#88

Post by dontreact »

dontreact wrote: post #55
chestervonwinch wrote: post #29

Is it possible to use the "deep dream" methods with a network trained for audio such as this? I wonder what that would sound like, e.g., beginning with a speech signal and enhancing with a network trained for music or vice versa.

We tried this but with less success than what wavenet did. https://wp.nyu.edu/ismir2016/wp-content/uploads/sites/2294/2...

There is a link to examples at the end

Re: WaveNet: A Generative Model for Raw Audio

#89

Post by Houshalter »

barrkel wrote: post #68
zardo wrote: post #42

Earlier quoted context omitted.

Audio quality does leave something to be desired. https://vimeo.com/47987691

Lincoln died before Edison invented the phonograph. That's a hoax.

Lincoln died in 1865, but the oldest recordings are from the 1860s. The video is definitely a hoax (http://www.firstsounds.org/research/others/lincoln.php), but it's at least theoretically possible his voice could have been recorded. In fact I believe there are some even older recordings from the 1850s, but I don't think those have been successfully recovered yet.

These early recordings are incredibly crude, and they did not have the technology at the time to play them back. They were just experiments in trying to view sound waves, not attempts to preserve information for future generations.

Re: WaveNet: A Generative Model for Raw Audio

#90

Post by Houshalter »

ycombinatorMan wrote: post #87
syllogism wrote: post #82

Earlier quoted context omitted.

Not really. They're training directly on the waveform, so the model can learn intonation. They just need to train on longer samples, and perhaps augment their linguistic representation with some extra discourse analysis. A big problem with generating prosody has always been that our theories of it don't really provide a great prediction of people's behaviours. It's also very expensive to get people to do the prosody…

theres 0 chance of effective intonation and tone without understanding of the material

And who says it can't understand the material? There have been recurrent networks trained that can translate between languages, or predict the next word in a sentence, at remarkable accuracy. Combined with wavenet this could be quite effective.
Post reply on HN