Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

WaveNet: A Generative Model for Raw Audio

deepmind.com

Post reply on HN
Print view View original

111–120 of 153 posts

Re: WaveNet: A Generative Model for Raw Audio

#111

Post by posterboy »

erichocean wrote: post #14

This can be used to implement seamless voice performance transfer from one speaker to another: 1. Train a WaveNet with the source speaker. 2. Train a second WaveNet with the target speaker. Or for something totally new, train a WaveNet with a bunch of different speakers until you get one you like. This becomes the target WaveNet . 3. Record raw audio from the source speaker. Fun fact: any algorithmic process that "re…

Thanks for the tl;dr. However, the fun fact is not true for surjective functions, IIRC, in which case multiple inputs may relate to one output, if this is relevant for WaveNets.

Re: WaveNet: A Generative Model for Raw Audio

#113

Post by AstralStorm »

JonnieCache wrote: post #64
itcrowd wrote: post #62

Earlier quoted context omitted.

Another fun fact: this actually happens with (cell) phone calls. You don't send your speech over the line, instead you send some parameters over the line which are then, at the receiving end, fed into a white(-ish) noise generator to recover the speech. Edit: not by using a neural net or deep learning, of course.

In case anyone is wondering, the technique is called linear predictive coding.

Predictive coding. Linear is a specific variant of it used in older codecs.

Re: WaveNet: A Generative Model for Raw Audio

#114

Post by mirimir »

erichocean wrote: post #27
swsieber wrote: post #24

Earlier quoted context omitted.

Oh, pair this with facial mapping[1] and you pretty much have an "impersonate any famous person" system. [1] http://www.graphics.stanford.edu/~niessner/thies2016face.htm...

Yup, I work in virtual filmmaking and there are tons of way to use this stuff. I give us 10-15 years before it's not possible to trust anything you see or hear that's recorded.

Or transmitted from one place to another :(

Re: WaveNet: A Generative Model for Raw Audio

#115

Post by creshal »

erichocean wrote: post #14

This can be used to implement seamless voice performance transfer from one speaker to another: 1. Train a WaveNet with the source speaker. 2. Train a second WaveNet with the target speaker. Or for something totally new, train a WaveNet with a bunch of different speakers until you get one you like. This becomes the target WaveNet . 3. Record raw audio from the source speaker. Fun fact: any algorithmic process that "re…

Sounds like a very fancy way to do compression with a massive custom dictionary.

Re: WaveNet: A Generative Model for Raw Audio

#116

Post by svantana »

erichocean wrote: post #14

This can be used to implement seamless voice performance transfer from one speaker to another: 1. Train a WaveNet with the source speaker. 2. Train a second WaveNet with the target speaker. Or for something totally new, train a WaveNet with a bunch of different speakers until you get one you like. This becomes the target WaveNet . 3. Record raw audio from the source speaker. Fun fact: any algorithmic process that "re…

> Fun fact: any algorithmic process that "renders" something given a set of inputs can be "run in reverse"

Now wait a minute, most algorithms cannot be run in reverse! The only general way to reverse an algo is to try all possible inputs, which has exponential complexity. That's the basis of RSA encryption. Maybe you're thinking about automatic differentiation, a general algo to get the gradient of the output w.r.t. the inputs. That allows you to search for a matching input using gradient descent, but that won't give you an exact match for most interesting cases (due to local minima).

I'm not trying to nitpick -- in fact I believe that IF algos were reversible then human-level AI would have been solved a long time ago. Just write a generative function that is capable of outputting all possible outputs, reverse, and inference is solved.

Re: WaveNet: A Generative Model for Raw Audio

#117

Post by benanne »

ThePhysicist wrote: post #36
augustl wrote: post #31

The music examples are utterly fascinating. It sounds insanely natural. The only thing I can hear that sounds unnatural, is the way that the reverberation in the room (the "echo") immediately gets lower when the raw piano sound itself gets lower. In a real room, if you produce a loud sound and immediately after a soft sound, the reverberation of the loud sound remains. But since this network only models "the sound ri…

I agree, the samples sound very natural. I ask myself though how similar they are to the data that has been used for training, as it would be trivial to rearrange individual pieces of a large training set in ways that sound good (especially if a human selects the good samples for presentation afterwards). What I'd really like to see therefore is a systematic comparison of the generated music to the training set, idea…

A nice property of the model is that it is easy to compute exact log-likelihoods for both training data and unseen data, so one can actually measure the degree of overfitting (which is not true for many other types of generative models). Another nice property of the model is that it seems to be extremely resilient to overfitting, based on these measurements.

Re: WaveNet: A Generative Model for Raw Audio

#118

Post by syllogism »

ycombinatorMan wrote: post #87
syllogism wrote: post #82

Earlier quoted context omitted.

Not really. They're training directly on the waveform, so the model can learn intonation. They just need to train on longer samples, and perhaps augment their linguistic representation with some extra discourse analysis. A big problem with generating prosody has always been that our theories of it don't really provide a great prediction of people's behaviours. It's also very expensive to get people to do the prosody…

theres 0 chance of effective intonation and tone without understanding of the material

I think your use of the term "understanding" is very unhelpful here. It's better to think about what you need to condition on to predict correctly.

In fact most intonation decisions are pretty local, within a sentence or two. The most important thing are given/new contrasts, i.e. the information structure. This is largely determined by the syntax, which we're doing pretty well at predicting, and which latent representations in a neural network can be expected to capture adequately.

Re: WaveNet: A Generative Model for Raw Audio

#119

Post by spiritus_ »

thomasahle wrote: post #108
Houshalter wrote: post #90

Earlier quoted context omitted.

And who says it can't understand the material? There have been recurrent networks trained that can translate between languages, or predict the next word in a sentence, at remarkable accuracy. Combined with wavenet this could be quite effective.

There could be cases where the intonation is dependent on things entirely outside of the book. If say a politician does something in the writing that is far from what we would expect them to do in today's world.

Like traditional audio books can capture perfectly what you're referring to...

Re: WaveNet: A Generative Model for Raw Audio

#120

Post by mdup »

posterboy wrote: post #111
erichocean wrote: post #14

This can be used to implement seamless voice performance transfer from one speaker to another: 1. Train a WaveNet with the source speaker. 2. Train a second WaveNet with the target speaker. Or for something totally new, train a WaveNet with a bunch of different speakers until you get one you like. This becomes the target WaveNet . 3. Record raw audio from the source speaker. Fun fact: any algorithmic process that "re…

Thanks for the tl;dr. However, the fun fact is not true for surjective functions, IIRC, in which case multiple inputs may relate to one output, if this is relevant for WaveNets.

Nitpicking: surjective functions do not relate to unicity of ouptuts; you'd rather talk about non-injective functions. I agree with your point, though.

(surjective != non-injective, in the same way that non-increasing != decreasing)

Post reply on HN