Explaining the SDXL Latent Space

Post by ttul » Thu, Feb 08, 2024, 7:16 AM UTC

wruza wrote: post #23

ttul wrote: post #3

I’ve been playing with diffusion a ton for the past few months, writing a new sampler that implements an iterative blending technique described in a recent paper. The latent space is rich in semantic information, so it can be a great place to apply various transformations rather than operating on the image directly. Yet it still has a significant spatial component, so things you do in one spatial area will affect tha…

Entered this thread to write your comment. I find SDXL inferior to 1.5 and yes, much harder to work with. My another issue is that sdxl images that you can see on the web always have that “from a movie/ads”-?ish? coating. Can’t explain it, but it feels even more uncanny than 1.5. SDXL is too resource-hungry for what it produces. 3x+ model sizes, 12GB vram is barely enough for it, 40 steps is the minimum, and I don’t…

Post by incrudible » Thu, Feb 08, 2024, 7:03 PM UTC

nomel wrote: post #28

incrudible wrote: post #27

Earlier quoted context omitted.

Does it lead to better results though? For the system, the best representation would be one that it learned - which is the latent representation, 4 channels in this case. Would it learn a "better" representation when fed with HSL instead of RGB? If so, what's the intuition? RGB somewhat resembles human vision, whereas HSL exists for interactive editing, and YCbCr exists for compression. If anything, I would expect YC…

> If so, what's the intuition? HSV closer resembles physical properties, for most natural things. Hue and saturation variations are usually meaningful variations in the actual material. Brightness variations often end up being mostly about lighting, rather than the material. It can be surprisingly effective for simple segmentation [1], which is why it's usually the first one implemented in computer vision classes. Ou…