11–20 of 44 posts
Author here, when I tried to understand diffusion models I realized that the code and math can be greatly simplified, which led to me writing this blog post and diffusion library. Happy to answer any questions.
Author here, when I tried to understand diffusion models I realized that the code and math can be greatly simplified, which led to me writing this blog post and diffusion library. Happy to answer any questions.
Your `get_sigma_embeds(batches, sigma)` seems to not use its first input? Did you mean to broadcast sigma to shape (batches, 1)?
[1] https://yang-song.net/blog/2021/score/
[2] https://lilianweng.github.io/posts/2021-07-11-diffusion-mode...
All machine learning models are convolutions, mark my words.
I have trouble seeing reinforcement learning as convolution, for example.
Author here, when I tried to understand diffusion models I realized that the code and math can be greatly simplified, which led to me writing this blog post and diffusion library. Happy to answer any questions.
FWIW, a friend of mine (and not "a friend of mine") wrote this minimal diffusion awhile back that I've found useful that's a bit more "full" w.r.t DDPM. More dropping here since I saw others excited to see code and it could provide good synergy here: https://github.com/VSehwag/minimal-diffusion/
oh this has code! great stuff. diffusion papers are famous for a lot of equations ( https://twitter.com/cto_junior/status/1766518604395155830 ) but code is much more legible (and precise?) for the rest of us. all theory papers should come with reference impl code. i'd love an extension of this for the diffusion transformer, which drives Sora and other videogen models. maybe combine this post with https://jaykmody.com…
On the other hand, if you want to really dig in, I'd suggest reading into works by Kingma, Gao, Ricky Tian Qi Chen, and honestly, any of Max Welling's students (Tomczak (was post doc), Hoogeboom, etc), and of course the unsung hero Aapo Hyvärinen. Here's a taste at a Kingma & Gao work that's on the lighter side but relevant to the SD3 paper. The unfortunate part is that there's a lot of reliance on knowing and understanding prior works which make these less approachable, but honestly this is a bit difficult to call a meaningful critique (it's research, not educational work aimed at public).
Good article, but I feel that it misses an important property of diffusion models that they model the score function (derivative of log prob) [1] and that diffusion sampling is akin to Langevin dynamics [2]. IMO these explain why it's easier to train these models than GANs, because of an easier modeling objective. [1] https://yang-song.net/blog/2021/score/ [2] https://lilianweng.github.io/posts/2021-07-11-diffusion-m…
To gain a deeper understanding of diffusion models, I encourage everyone to read all of these blog posts and learn about the different interpretations :)
oh this has code! great stuff. diffusion papers are famous for a lot of equations ( https://twitter.com/cto_junior/status/1766518604395155830 ) but code is much more legible (and precise?) for the rest of us. all theory papers should come with reference impl code. i'd love an extension of this for the diffusion transformer, which drives Sora and other videogen models. maybe combine this post with https://jaykmody.com…
They're famous for lots of equations, but truth be told, most diffusion researchers I know have the exact same response. A lot of people repeat the same exact equations and they only are there for review purposes. On the other hand, if you want to really dig in, I'd suggest reading into works by Kingma, Gao, Ricky Tian Qi Chen, and honestly, any of Max Welling's students (Tomczak (was post doc), Hoogeboom, etc), and…