Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Illustrated Stable Diffusion

jalammar.github.io

Post reply on HN
Print view View original

11–20 of 128 posts

Re: The Illustrated Stable Diffusion

#11

Post by minimaxir »

netruk44 wrote: post #9
Waterluvian wrote: post #5

Closer. But I still get lost when words like “tensor” are used. “structured lists of numbers” really doesn’t seem to explain it usefully. This reminds me that explaining seemingly complex things in simple terms is one of the most valuable and rarest skills in engineering. Most people just can’t. And often because they no-longer remember what’s not general knowledge. You end up with a recursive Feynmannian “now explai…

If it helps you to understand at all, assuming you have a CS background, any time you see the word "tensor" you can replace it with "array" and you'll be 95% of the way to understanding it. Or "matrix" if you have a mathematical background. Whereas CS arrays tend to be 1 dimensional, and sometimes 2 dimensional, tensors can be as many dimensions as you need. A 256x256 photo with RGB channels would be stored as a [256…

A more practical example of the added dimensionality of tensors is the addition of a batch dimension, so a 8 image batch per training step would be a (8, 256, 256, 3) tensor.

Tools such as PyTorch's DataLoader can efficiently collate multiple inputs into a batch.

Re: The Illustrated Stable Diffusion

#13

Post by coldcode »

I find SD to be amazing technology, but it still (mostly) sucks at producing "intelligent" images. It basically fancy math that turns noise into images (from the opposite it trained on) but still has no idea what it is producing. If you run it long enough you eventually get lucky and find a gem. I like to try "George Washington riding a Unicorn in Times Square"; I've so far never gotten anything a first year art student can draw. I wonder how long it will take before something more "AI" than "ML" will have an understanding even close to what a simple human brain can process.

In the meantime it's fun to play with it, plus I'd like to better understand the noise training process.

Re: The Illustrated Stable Diffusion

#14

Post by minimaxir »

coldcode wrote: post #13

I find SD to be amazing technology, but it still (mostly) sucks at producing "intelligent" images. It basically fancy math that turns noise into images (from the opposite it trained on) but still has no idea what it is producing. If you run it long enough you eventually get lucky and find a gem. I like to try "George Washington riding a Unicorn in Times Square"; I've so far never gotten anything a first year art stud…

With SD, you have to use modifier quality/positional/artist keywords, as vanilla inputs give the model too much freedom.

Re: The Illustrated Stable Diffusion

#15

Post by yieldcrv »

What are you guys currently using for Stable Diffusion on OSX with M1?

There are so many variants and forks that I don't know which one to install any more. Something that takes advantage of Metal and the CPU cores.

Any that retains the "upload a sketch and then add a description" feature?

Re: The Illustrated Stable Diffusion

#17

Post by pram »

yieldcrv wrote: post #15

What are you guys currently using for Stable Diffusion on OSX with M1? There are so many variants and forks that I don't know which one to install any more. Something that takes advantage of Metal and the CPU cores. Any that retains the "upload a sketch and then add a description" feature?

I'm using InvokeAI. Follow the instructions and it will work flawlessly.

https://github.com/invoke-ai/InvokeAI

Re: The Illustrated Stable Diffusion

#18

Post by Waterluvian »

netruk44 wrote: post #9
Waterluvian wrote: post #5

Closer. But I still get lost when words like “tensor” are used. “structured lists of numbers” really doesn’t seem to explain it usefully. This reminds me that explaining seemingly complex things in simple terms is one of the most valuable and rarest skills in engineering. Most people just can’t. And often because they no-longer remember what’s not general knowledge. You end up with a recursive Feynmannian “now explai…

If it helps you to understand at all, assuming you have a CS background, any time you see the word "tensor" you can replace it with "array" and you'll be 95% of the way to understanding it. Or "matrix" if you have a mathematical background. Whereas CS arrays tend to be 1 dimensional, and sometimes 2 dimensional, tensors can be as many dimensions as you need. A 256x256 photo with RGB channels would be stored as a [256…

This helps. Thank you. Any advice on where to look to understand why the word tensor was used?

Re: The Illustrated Stable Diffusion

#19

Post by jw1224 »

coldcode wrote: post #13

I find SD to be amazing technology, but it still (mostly) sucks at producing "intelligent" images. It basically fancy math that turns noise into images (from the opposite it trained on) but still has no idea what it is producing. If you run it long enough you eventually get lucky and find a gem. I like to try "George Washington riding a Unicorn in Times Square"; I've so far never gotten anything a first year art stud…

> "George Washington riding a Unicorn in Times Square"

The “secret” to Stable Diffusion (and other CLIP-based models) is being as descriptive as possible. This prompt, whilst easy for humans to imagine, actually has a whole lot of ambiguity baked in.

How high is the unicorn flying? Is the unicorn even flying, or on the ground? How old is George Washington? What visual style is the image in? Is the image from the perspective of a pedestrian at ground level, or from up at skyscraper level?

The more ambiguous the prompt, the less cohesive the image.

To demonstrate, here’s 4 renders from your original prompt: https://imgur.com/a/Jo4qfOp

And here’s 4 using the prompt “George Washington riding a unicorn in Times Square, cinematic composition, concept art, digital illustration, detailed”: https://imgur.com/a/lB36JqC

Certainly not perfect, but for an additional 15 seconds of effort, far better.

Post reply on HN