Viewing profile — benanne

Thanks for pointing this out! I'm not sure why, the files are still on my Dropbox, they must have changed the link format at some point? I've gone ahead and fixed them.

It kind of does! In the modern era of generative modelling, it seems like we rely on pre-training to capture the data distribution, and then on post-training (and various other tri…

I briefly covered that connection in an earlier blog post: https://sander.ai/2023/07/20/perspectives.html#flow ... but it's definitely something that might deserve a longer-form tr…

I actually wrote down some thoughts about audio phase in a previous blog post: https://sander.ai/2020/03/24/audio-generation.html#motivatio... I have an example audio clip in there…

Sorry to hear that. My blog posts are intended to build intuition. I also write academic papers, which of course involves a different standard of rigour. Perhaps you'd prefer those…

I'm not sure if frequency decomposition makes sense for anything that's not grid-structured, but there is certainly evidence that there is positive "transfer" between generative mo…

Oof, you're not going to like this other blog post I wrote then :D https://sander.ai/2023/07/20/perspectives.html

Thanks for reading! Absolutely, I included a few references that explore that approach at the bottom of section 4 (last two paragraphs).

Thanks for reading! The paper that directly inspired this blog post actually investigates the latter (blurring as the corruption process): https://arxiv.org/abs/2206.13397

Thanks for reading! Check out subspace diffusion: https://arxiv.org/abs/2205.01490

I've since moved on to work primarily on diffusion models, so I have a series of blog posts about that topic as well! - https://sander.ai/2022/01/31/diffusion.html is about the lin…

In Georgian, mother is "deda" and father is "mama", which lends further evidence to this :)

A nice property of the model is that it is easy to compute exact log-likelihoods for both training data and unseen data, so one can actually measure the degree of overfitting (whic…

Agreed. Theano has trouble dealing efficiently with very deeply nested graphs.

Why wouldn't this work in Theano? >>> import theano >>> import theano.tensor as T >>> state = theano.shared(1.0) >>> states = [] >>> for step in range(10): >>> state = state + stat…

Nobody forced us to open-source Lasagne, so I think that remark was a bit unfair. If we really didn't care about anything but graduating, why would we bother going through the trou…

I have some doubts about this. Deep learning moves fast and DBNs are pretty much outdated models, even for unsupervised pre-training. It doesn't make much sense to me that unsuperv…

I feel like this guide comes about 5 years too late - RBMs as density models have been shown to be relatively weak, except in the case of binary data. For continuous data, you can …

Not to my knowledge. For speech it should be sort of doable to get enough data to train a large net and then do inceptionism-style stuff with it. For music, gathering enough traini…