Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A Web UI for Stable Diffusion

github.com

Post reply on HN
Print view View original

31–40 of 148 posts

Re: A Web UI for Stable Diffusion

#31

Post by juliendorra »

jrm4 wrote: post #17

So, and this is an ELI5 kind of question I suppose. There must be something going on like "processing a kazillion images" and I'm trying to wrap my head around how (or what part of) that work is "offloaded" to your home computer/graphics card? I just can't seem to make sense of how you can do it at home if you're not somehow in direct contact with "all the data?" e.g. must you be connected to the internet, or "stable…

That’s the interesting part: all the images generated are derived from a less than 4gb model (the trained weights of the neural network).

So in a way, hundreds of billions of possible images are all stored in the model (each a vector in multidimensional latent space) and turned into pixels on demand (drived by the language model that knows how to turn words into a vector in this space)

As it’s deterministic (given the exact same request parameters, random seed included, you get the exact same image) it’s a form of compression (or at least encoding decoding) too: I could send you the parameters for 1 million images that you would be able to recreate on your side, just as a relatively small text file.

Re: A Web UI for Stable Diffusion

#32

Post by CitrusFruits »

totoglazer wrote: post #24
CitrusFruits wrote: post #14

Earlier quoted context omitted.

I'm using it with a 2070 (4 year old card with 8gb vram) and it takes about 5 seconds for a 512x512 image. It's been plenty fast to have some fun, but I think I'd want faster if it was part of a professional work flow.

What settings? That seems faster than expected.

It was the defaults for the webui I used. Faster than I expected too, but the results were all legit.

Edit: Got home and was able to double check. It's actually a solid 10 seconds per image with the following settings: seed:466520488 width:512 height:512 steps:50 cfg_scale:7.5 sampler:k_lms. Still quick enough for some fun, but could be annoying if you're need to do multiple iterations a minute.

Re: A Web UI for Stable Diffusion

#34

Post by bastawhiz »

amelius wrote: post #33

Regarding the opening image: if it can't correctly put the marks on dice, how can it put eyes, nose and mouth correctly on a human face?

Presumably the number of faces in the training set far exceeds the number of dice by more than a few orders of magnitude.

Re: A Web UI for Stable Diffusion

#35

Post by amelius »

juliendorra wrote: post #31
jrm4 wrote: post #17

So, and this is an ELI5 kind of question I suppose. There must be something going on like "processing a kazillion images" and I'm trying to wrap my head around how (or what part of) that work is "offloaded" to your home computer/graphics card? I just can't seem to make sense of how you can do it at home if you're not somehow in direct contact with "all the data?" e.g. must you be connected to the internet, or "stable…

That’s the interesting part: all the images generated are derived from a less than 4gb model (the trained weights of the neural network). So in a way, hundreds of billions of possible images are all stored in the model (each a vector in multidimensional latent space) and turned into pixels on demand (drived by the language model that knows how to turn words into a vector in this space) As it’s deterministic (given th…

> As it’s deterministic (given the exact same request parameters, random seed included, you get the exact same image) it’s a form of compression (or at least encoding decoding) too: I could send you the parameters for 1 million images that you would be able to recreate on your side, just as a relatively small text file.

For any input image? Or do you mean an image generated by the model?

Re: A Web UI for Stable Diffusion

#36

Post by amelius »

bastawhiz wrote: post #34
amelius wrote: post #33

Regarding the opening image: if it can't correctly put the marks on dice, how can it put eyes, nose and mouth correctly on a human face?

Presumably the number of faces in the training set far exceeds the number of dice by more than a few orders of magnitude.

In one of the other posts I noticed this option:

> GFPGAN Face Correction: Automatically correct distorted faces with a built-in GFPGAN option, fixes them in less than half a second

So apparently there is still an issue with faces.

Re: A Web UI for Stable Diffusion

#37

Post by juliendorra »

amelius wrote: post #35
juliendorra wrote: post #31

Earlier quoted context omitted.

That’s the interesting part: all the images generated are derived from a less than 4gb model (the trained weights of the neural network). So in a way, hundreds of billions of possible images are all stored in the model (each a vector in multidimensional latent space) and turned into pixels on demand (drived by the language model that knows how to turn words into a vector in this space) As it’s deterministic (given th…

> As it’s deterministic (given the exact same request parameters, random seed included, you get the exact same image) it’s a form of compression (or at least encoding decoding) too: I could send you the parameters for 1 million images that you would be able to recreate on your side, just as a relatively small text file. For any input image? Or do you mean an image generated by the model?

I meant images generated by the model. Now that I think of it I could just send you the sampled vectors and you could feed that to the vector to image part.

Re: A Web UI for Stable Diffusion

#39

Post by hombre_fatal »

totoglazer wrote: post #23
drexlspivey wrote: post #11

Earlier quoted context omitted.

How is M1/M2 support for SD? Is there a significant performance drop? Presumably you would be able to buy a 32GB M2 and be future proof because of the shared memory between CPU/GPU.

There was a long thread last week. It’s honestly pretty good if you follow the instructions. 30-40 seconds/image.

Yeah, I followed the instructions on a M1 Macbook Pro (Monterey 12.5.1) and it worked without extra effort. 30-40 seconds per image. I have 32GB but image generation doesn’t even use half of it. The hard part has been to generate prompts that do what I want.

Re: A Web UI for Stable Diffusion

#40

Post by ShamelessC »

amelius wrote: post #33

Regarding the opening image: if it can't correctly put the marks on dice, how can it put eyes, nose and mouth correctly on a human face?

> Regarding the opening image: if it can't correctly put the marks on dice, how can it put eyes, nose and mouth correctly on a human face?

It helps if you consider it all as effectively advanced compression. Everything the model can do is limited by its architecture, the number of parameters in the model, and the accuracy and size of the training data.

The underlying architecture is a transformer (e.g. GPT3) wired to a (denoising) diffusion model.

Current flaws with this approach:

- Transformers seem to approach a "bag-of-words" model, often ignoring the ordering of the words. Among other things, this means that text-to-image models are very bad at "binding attributes" [0]. This is why "a boy wearing a red shirt and a girl wearing a black jacket" may fail (putting the colors on the wrong items, for instance).

- Autoregressive transformers have no means to correct early mistakes.

- Training data is captioned images and the captions are likely noisy and under-specified. Every time it sees a face labeled as "face" - it tries to generate a face from the distribution of _all_ faces in the data. The same goes for the dice. If the dice are just labeled "dice", but don't have a description of how they landed - the model has to guess which angle you're referring to. As a sibling comment points out, this is exacerbated by the relative frequencies of examples of the data in the dataset.

[0] https://wikiless.org/wiki/Binding_(linguistics)

Post reply on HN