https://jalammar.github.io/images/stable-diffusion/article-F... Can you say with a straight face that this image (from the original paper) was intended to explain rather than obfuscate?
If you're in AI research, that's a pretty standard diagram
The Illustrated Stable Diffusion
71–80 of 128 posts
Re: The Illustrated Stable Diffusion
#72Great overview, I think the part for me which is still very unintuitive is the denoising process. If the diffusion process is removing noise by predicting a final image and comparing it to the current one, why can't we just jump to the final predicted image? Or is the point that because its an iterative process, each noise step results in a different "final image" prediction?
In the reverse diffusion process, the reason we can't directly jump from a noisy image at step t to a clean image at step 0 is that each possible noisy image at step t may be visited by potentially many real images during the forward diffusion process. Thus, our model which inverts the diffusion process by minimizing least-squares prediction error of a clean image given a noisy image at step t will learn to predict t…
This is the best explanation I have ever read - on any topic.
Re: The Illustrated Stable Diffusion
#73https://jalammar.github.io/images/stable-diffusion/article-F... Can you say with a straight face that this image (from the original paper) was intended to explain rather than obfuscate?
Yes, I can read it perfectly fine. It relies on a lot of notation that AI researchers are familiar with.
[deleted]
Re: The Illustrated Stable Diffusion
#74Earlier quoted context omitted.
How are sum-types handled in deep learning? E.g., a type that holds "a 3x2 tensor OR a 4x6x2x5 tensor".
I'm not aware of tensor shape type safety (in general) being widely used in deep learning, let alone sum-types. I believe Pytorch and TensorFlow lack support for tensor shape type checking (looks like there is a Pytorch issue open: https://github.com/pytorch/pytorch/issues/26889 ).
https://docs.rs/ndarray/0.15.6/ndarray/
Rust to the rescue, as it always is for safety. (I don't know if this directly translates for tensors, I used it for manipulating point clouds and iirc it allows for arbitrary dimension containers)
Re: The Illustrated Stable Diffusion
#75https://jalammar.github.io/images/stable-diffusion/article-F... Can you say with a straight face that this image (from the original paper) was intended to explain rather than obfuscate?
It's as clear as wiring diagram for me. But I read a lot of ML literature.
Re: The Illustrated Stable Diffusion
#76Earlier quoted context omitted.
I updated the post to say "multi-dimensional array". In a context like this, we use tensor because it allows for any number of dimensions (while vector/ array is only one, matrix is two). When you get into ML libraries, both popular packages PyTorch and TensorFlow use the "tensor" terminology. It's a good point. Hope it's clearer for devs with "array" terminology.
> we use tensor because it allows for any number of dimensions "Vector" implies one dimension and "matrix" strongly implies two. But an array can have any number of dimensions, so "array" is the best word. We don't need the word "tensor"; when the context is programming, "tensor" is only confusing and doesn't really add any useful meaning.
"Array" implies one dimension to me. Maybe n-array?
Re: The Illustrated Stable Diffusion
#77I find SD to be amazing technology, but it still (mostly) sucks at producing "intelligent" images. It basically fancy math that turns noise into images (from the opposite it trained on) but still has no idea what it is producing. If you run it long enough you eventually get lucky and find a gem. I like to try "George Washington riding a Unicorn in Times Square"; I've so far never gotten anything a first year art stud…
> "George Washington riding a Unicorn in Times Square" The “secret” to Stable Diffusion (and other CLIP-based models) is being as descriptive as possible. This prompt, whilst easy for humans to imagine, actually has a whole lot of ambiguity baked in. How high is the unicorn flying? Is the unicorn even flying, or on the ground? How old is George Washington? What visual style is the image in? Is the image from the pers…
> How high is the unicorn flying? Is the unicorn even flying, or on the ground?
For what it's worth, I think you have mixed up unicorns and pegasus!
Re: The Illustrated Stable Diffusion
#78Kudos to Author for writing this. The author's other guide, https://jalammar.github.io/gentle-visual-intro-to-data-analy... is THE most approachable introduction to Pandas for Excel users like me who are "comfortable" with Python(disclaimer: Self taught EE/Hardware guy). I will definitely spend some time this weekend to go over this tome. Thanks again.
Re: The Illustrated Stable Diffusion
#79Another very good (although less deep) explanation was published yesterday by Computerphile:
Re: The Illustrated Stable Diffusion
#80What are you guys currently using for Stable Diffusion on OSX with M1? There are so many variants and forks that I don't know which one to install any more. Something that takes advantage of Metal and the CPU cores. Any that retains the "upload a sketch and then add a description" feature?
I just went ahead and got myself a PC with a good graphics card.