Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Stable Diffusion XL 1.0

techcrunch.com

Post reply on HN
Print view View original

101–110 of 182 posts

Re: Stable Diffusion XL 1.0

#101

Post by cheald »

mt3ck wrote: post #99

Is there anything like this for the vector landscape? This may just be due to the iterative denoising approach a lot of these models take but they only seem to work well when creating raster style images. In my experience when you ask them to create logos, shirt designs, illustrations, they tend to not work as well and introduce a lot of artifacts, distortions, incorrect spellings etc.

A lot of people are having success by adding extra networks (lora is the most common) which are trained on the type of image you're looking for. It's still a raster image, of course, but you can produce images which look very much like rasterizations of vector images, which you can then translate back into SVGs in Inkscape or similar.

Re: Stable Diffusion XL 1.0

#104

Post by orbital-decay »

mt3ck wrote: post #99

Is there anything like this for the vector landscape? This may just be due to the iterative denoising approach a lot of these models take but they only seem to work well when creating raster style images. In my experience when you ask them to create logos, shirt designs, illustrations, they tend to not work as well and introduce a lot of artifacts, distortions, incorrect spellings etc.

If you mean raster images that look like vector and contain arbitrary text and shapes, controlnets/T2I adapters do work for this. You could train your custom controlnet for this, too. (it requires understanding)

As for directly generating vector images, there's nothing yet. Your best bet is generating vector-looking raster and tracing it.

Re: Stable Diffusion XL 1.0

#105

Post by ShamelessC »

airgapstopgap wrote: post #85
amilios wrote: post #61

I always wondered why the vision models don't seem to be following the whole "scale up as much as possible" mantra that has defined the language models of the past few years (to the same extent). Even 3.5 billion parameters is absolutely nothing compared to the likes of GPT-3, 3.5, 4, or even the larger open-source language models (e.g. LLaMA-65B). Is it just an engineering challenge that no one has stepped up for ye…

Diffusion is more parameter-efficient and you quickly saturate the target fidelity, especially with some refiner cascade. It's a solved problem. You do not need more than maybe 4B total. Images are far more redundant than text. In fact, most interesting papers since Imagen show that you get more mileage out of scaling the text encoder part, which is, of course, a Transformer. This is what drives accuracy, text render…

> Seeing as you can throw out diffusion altogether and synthesize images with transformers [3]

That’s actually how this whole party got started. DALL-E (the first one) was a transformer model trained on image tokens from an early VAE (and text tokens ofc). Researchers from CompVis developed VQGAN in response. OpenAI showed improved fidelity with guided diffusion over ImageNet (classes) and subsequently DALLE2 using pixel space diffusion and cascading up sampling. CompVis responded with Latent Diffusion which used diffusion in the latent space of some new VQGANs.

The paper you mention is interesting! They go back to the DALL-E 1 method but train two VQGAN’s for upsampling and increase the parameter count. This is faster, but only faster than originally reported benchmarks using inferior sampling methods for their diffusion. I would be curious if they can beat some of the more recent ones which require as few as 10-20 steps.

They also improve on FID/CLIP scores likely by using more parameters. This might be a memory/time trade off though. I would be curious how much more VRAM their model requires compared to SD, MJ, Kandinsky.

The same goes for using T5-XXL. You’ll win FID score contests but no one will be able to run it without an A100 or TPU pod.

Re: Stable Diffusion XL 1.0

#106

Post by elpocko »

Etherlord87 wrote: post #98
airgapstopgap wrote: post #85

Earlier quoted context omitted.

Diffusion is more parameter-efficient and you quickly saturate the target fidelity, especially with some refiner cascade. It's a solved problem. You do not need more than maybe 4B total. Images are far more redundant than text. In fact, most interesting papers since Imagen show that you get more mileage out of scaling the text encoder part, which is, of course, a Transformer. This is what drives accuracy, text render…

> Images are far more redundant than text. "A picture is worth a thousand words" - I wonder how (in)accurate this popular saying turned out to be? :D

I'm gonna go ahead and say in 2023, one detailed picture (512x512) is worth about 30 words.

Re: Stable Diffusion XL 1.0

#107

Post by k12sosse »

seydor wrote: post #34
Der_Einzige wrote: post #20

Earlier quoted context omitted.

It's already supported in automatic1111 (see recent updates), and someone in the community will convert it to the automatic1111 format within minutes/hours after it's released on huggingface.

whats the memory usage of sdxl ?

Runs great on my 10GB 3080 FTW3. ComfyUI moreso than auto1111.

Re: Stable Diffusion XL 1.0

#108

Post by SketchySeaBeast »

elpocko wrote: post #106
Etherlord87 wrote: post #98

Earlier quoted context omitted.

> Images are far more redundant than text. "A picture is worth a thousand words" - I wonder how (in)accurate this popular saying turned out to be? :D

I'm gonna go ahead and say in 2023, one detailed picture (512x512) is worth about 30 words.

I guess that depends on the prompt.

Re: Stable Diffusion XL 1.0

#109

Post by k12sosse »

latchkey wrote: post #46

Amazing that their examples at the bottom of the page still show really messed up human hands.

Hands being bad is a result of people one shotting images, you need to go repaint them afterwards I've found. But it'll do it great if you inpaint well.

Re: Stable Diffusion XL 1.0

#110

Post by k12sosse »

latchkey wrote: post #94
HelloMcFly wrote: post #91

Earlier quoted context omitted.

I've personally observed that the drawing of hands in Midjourney and SD has been getting incrementally better release after release.

That's why I'm amazed they picked images with totally borked up hands to put on their press release. Truth in advertising!

If they cherry pick the examples people would get the wrong idea. What I like about imagegen is your results are really only bound by your patience
Post reply on HN