Love the music part! Mmmh ... infinite jazz.
WaveNet: A Generative Model for Raw Audio
51–60 of 153 posts
Re: WaveNet: A Generative Model for Raw Audio
#52How much data does a model take up? I wonder if this would work for compression? Train a model on a corpus of audio, then store the audio as text that turns back into a close approximation of that audio. (Optionally store deltas for egregious differences.)
It would be a slow (but very efficient information-wise - only have to send text which itself can be compressed!) decompression process with current models / hardware due to sequential relationships in generation.
I am sure people will start trying to speed this up, as it could be a game changer in that space with a fast enough implementation. Google also has a lot of great engineers with direct motivation to get it working on phones, and a history of porting recent research in to the Android speech pipeline.
The results speak for themselves - step 1 is almost always "make it work" after all, and this works amazingly well! Step 2 or 3 is "make it fast", depending who you ask.
Re: WaveNet: A Generative Model for Raw Audio
#53The samples sound amazing. These causal convolutions look like a great idea, will have to re-read a few times. All the previous generative audio from raw audio samples I've heard (using LSTM) has been super noisy. These are crystal clear. Dilated convolutions are already implemented in TF, look forward to someone implementing this paper and publishing the code.
I did a review for PixelCNN as a part of my summer internship, it covers a bit about how careful masking can be used to create a chain of conditional probabilities [0], which AFAIK is exactly how this "causal convolution" works (can't have dependencies in the 'future'). The PixelCNN and PixelRNN papers also cover this in a fair bit of detail. Ishaan Gulrajani's code is also a great implementation reference for PixelC…
Heh, just read it! Very useful, will have to go through in detail
Re: WaveNet: A Generative Model for Raw Audio
#54Wow. I badly want to try this out with music, but I've taken little more than baby steps with neural networks in the past: am I stuck waiting for someone else to reimplement the stuff in the paper? IIRC someone published an OSS implementation of the deep dreaming image synthesis paper fairly quickly...
Re-implementation will be hard, several people (including me) have been working on related architectures, but they have a few extra tricks in WaveNet that seem to make all the difference, on top of what I assume is "monster scale training, tons of data".
The core ideas from this can be seen in PixelRNN and PixelCNN, and there are discussions and implementations for the basic concepts of those out there [0][1]. Not to mention the fact that conditioning is very interesting / tricky in this model, at least as I read it. I am sure there are many ways to do it wrong, and getting it right is crucial to having high quality results in conditional synthesis.
[0] https://github.com/tensorflow/magenta/blob/master/magenta/re...
[1] https://github.com/igul222/pixel_rnn/blob/master/pixel_rnn.p...
Re: WaveNet: A Generative Model for Raw Audio
#55Is it possible to use the "deep dream" methods with a network trained for audio such as this? I wonder what that would sound like, e.g., beginning with a speech signal and enhancing with a network trained for music or vice versa.
We tried this but with less success than what wavenet did. https://wp.nyu.edu/ismir2016/wp-content/uploads/sites/2294/2...
Re: WaveNet: A Generative Model for Raw Audio
#56Earlier quoted context omitted.
Oh, pair this with facial mapping[1] and you pretty much have an "impersonate any famous person" system. [1] http://www.graphics.stanford.edu/~niessner/thies2016face.htm...
Yup, I work in virtual filmmaking and there are tons of way to use this stuff. I give us 10-15 years before it's not possible to trust anything you see or hear that's recorded.
Really? I haven't trusted anything recorded in years.
Re: WaveNet: A Generative Model for Raw Audio
#57The music examples are utterly fascinating. It sounds insanely natural. The only thing I can hear that sounds unnatural, is the way that the reverberation in the room (the "echo") immediately gets lower when the raw piano sound itself gets lower. In a real room, if you produce a loud sound and immediately after a soft sound, the reverberation of the loud sound remains. But since this network only models "the sound ri…
That seems completely tractable by simply adding a bit of the right reverb to the generated sample, more or less "in post".
Re: WaveNet: A Generative Model for Raw Audio
#58The music examples are utterly fascinating. It sounds insanely natural. The only thing I can hear that sounds unnatural, is the way that the reverberation in the room (the "echo") immediately gets lower when the raw piano sound itself gets lower. In a real room, if you produce a loud sound and immediately after a soft sound, the reverberation of the loud sound remains. But since this network only models "the sound ri…
[deleted]
Re: WaveNet: A Generative Model for Raw Audio
#59Please make it sound like Morgan Freeman.
Re: WaveNet: A Generative Model for Raw Audio
#60The music examples are utterly fascinating. It sounds insanely natural. The only thing I can hear that sounds unnatural, is the way that the reverberation in the room (the "echo") immediately gets lower when the raw piano sound itself gets lower. In a real room, if you produce a loud sound and immediately after a soft sound, the reverberation of the loud sound remains. But since this network only models "the sound ri…
It shot me forward to a time where people just click a button to generate music they want to listen to. If you really like the generation, you save it and share it. It wouldn't have all of the other aspects that we derive from human-produced music like soul/emotion (because we know it's coming from a human, not because of how it sounds), but it would be a cool application idea anyway.