Live data from Hacker News

WaveNet: A Generative Model for Raw Audio

deepmind.com

121–130 of 153 posts

Re: WaveNet: A Generative Model for Raw Audio

#121

This can be used to implement seamless voice performance transfer from one speaker to another: 1. Train a WaveNet with the source speaker. 2. Train a second WaveNet with the target speaker. Or for something totally new, train a WaveNet with a bunch of different speakers until you get one you like. This becomes the target WaveNet . 3. Record raw audio from the source speaker. Fun fact: any algorithmic process that "re…

> Fun fact: any algorithmic process that "renders" something given a set of inputs can be "run in reverse" Now wait a minute, most algorithms cannot be run in reverse! The only general way to reverse an algo is to try all possible inputs, which has exponential complexity. That's the basis of RSA encryption. Maybe you're thinking about automatic differentiation, a general algo to get the gradient of the output w.r.t.…

This also makes me think of "inverse problems", in the context of mathematics, physics.

E.g. a forward problem might be to solve some PDE to simulate the state of a system from some known initial conditions.

The inverse problem could be to try to reverse engineer what the initial conditions were given the observed state of the system.

Inverse problems are typically much harder to deal with, and much harder to solve. E.g. perhaps they don't have a unique solution, or the solution is a highly discontinuous function of the inputs, which amplifies any measurement errors. In practice this can be addressed by regularisation aka introducing strong structural assumptions about what the expected solution should be like. This can be quite reasonable from a Bayesian perspective.

https://en.wikipedia.org/wiki/Inverse_problem#Mathematical_c...

Re: WaveNet: A Generative Model for Raw Audio

#122
post #31

The music examples are utterly fascinating. It sounds insanely natural. The only thing I can hear that sounds unnatural, is the way that the reverberation in the room (the "echo") immediately gets lower when the raw piano sound itself gets lower. In a real room, if you produce a loud sound and immediately after a soft sound, the reverberation of the loud sound remains. But since this network only models "the sound ri…

It shot me forward to a time where people just click a button to generate music they want to listen to. If you really like the generation, you save it and share it. It wouldn't have all of the other aspects that we derive from human-produced music like soul/emotion (because we know it's coming from a human, not because of how it sounds), but it would be a cool application idea anyway.

Have you tried https://www.jukedeck.com ? AI composed music at the touch of a button.

Re: WaveNet: A Generative Model for Raw Audio

#124

And when you think of all those Hollywood SF movies where the robot could reason and act quite well but in a tin-voice. How wrong they got it. We can simulate high quality voices but we can't have our reasoning, walking robots.

Depending on how you mean 'reasoning, walking robots' then not yet really.. but every few weeks or months another amazing deep learning/NN whatever thing comes out in different domains. So these types of techniques seem to have very broad application.

Of course, if you mean 'walking' in a literal sense, there are a number of impressive walking robots such as Atlas https://www.youtube.com/watch?v=rVlhMGQgDkY, HRP-2 https://www.youtube.com/watch?v=T6BSSWWV-60 or HRP 4C https://www.youtube.com/watch?v=YvbAqw0sk6M, etc.. Also there are many types of useful reasoning systems. I am guessing you are thinking of language understanding and generation.. but I believe these types of techniques are being applied quite impressively in that area also, from DeepMind or Watson https://www.youtube.com/watch?v=i-vMW_Ce51w etc.

Re: WaveNet: A Generative Model for Raw Audio

#126

Earlier quoted context omitted.

We tried this but with less success than what wavenet did. https://wp.nyu.edu/ismir2016/wp-content/uploads/sites/2294/2...

There is a link to examples at the end

Interesting! So if I understand correctly, much of the noise in the generated audio is due to the noise in the learned filters?

I assume some regularization is added to the weights during training, say L1 or L2? If this is the case, this essentially equivalent to assuming the weight values are distributed i.i.d. Laplacian or Gaussian. It seems you could learn less noisy filters by using a prior that assumes dependency between values within each filter, thereby enforcing smoothness or piecewise smoothness of each filter during training.

Re: WaveNet: A Generative Model for Raw Audio

#127

Earlier quoted context omitted.

It would be a slow (but very efficient information-wise - only have to send text which itself can be compressed!) decompression process with current models / hardware due to sequential relationships in generation. I am sure people will start trying to speed this up, as it could be a game changer in that space with a fast enough implementation. Google also has a lot of great engineers with direct motivation to get it…

We've known for decades that neural networks are really good at image and video compression. But as far as I know, this has never been used in practice, because the compression and decompression times are ridiculous. I imagine this would be even more true for audio.

The magic pony guys (who sold to twitter) have patents and implementations of a super resolution CNN for realtime video.

http://www.cv-foundation.org/openaccess/content_cvpr_2016/pa...

Re: WaveNet: A Generative Model for Raw Audio

#128

Earlier quoted context omitted.

theres 0 chance of effective intonation and tone without understanding of the material

I think your use of the term "understanding" is very unhelpful here. It's better to think about what you need to condition on to predict correctly. In fact most intonation decisions are pretty local, within a sentence or two. The most important thing are given/new contrasts, i.e. the information structure. This is largely determined by the syntax, which we're doing pretty well at predicting, and which latent represen…

The same sentence can have a very nonlocal difference in intonation.

Say, “They went in the shed”. You won't pronounce it in a neutral voice if it was explained in the previous chapter that a serial killer is in it.

On the other hand, if the shed contains a shovel that is quickly needed to dig out a treasure, which is the subject of the novel since page 1, you will imply urgency.

Re: WaveNet: A Generative Model for Raw Audio

#129
post #73

Wonder if there are any implications here for breaking (MitM) ZRTP protocol. https://en.wikipedia.org/wiki/ZRTP At some point to authenticate both parties verify a short message by reading it to each other. However, NSA has already tried to MitM that about 10 years ago by using voice synthesis. It was deemed inadequate at the time. Wonder if TTS improvements like these, change that game and make it more plausable sce…

This will make private in person key exchange way more important. Especially as the attack vector is so cheap (software).

Re: WaveNet: A Generative Model for Raw Audio

#130

I'm guessing DeepMind has already done this (or is already doing), but conditioning on a video is the obvious next step. It would be incredibly interesting to see how accurate it can get generating the audio for a movie. Though I imagine for really great results they'll need to mix in an adversarial network.

Oh yes, extract voice and intonation from one language, and then synthesize it in another language -> we get automated dubbing. Could also possibly try to lipsync.

[deleted]
Post reply on HN