Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Running Stable Diffusion on Your GPU with Less Than 10Gb of VRAM

constant.meiring.nz

Post reply on HN
Print view View original

21–30 of 141 posts

Re: Running Stable Diffusion on Your GPU with Less Than 10Gb of VRAM

#21

Post by barrkel »

I started out using my old GTX 1080 on Thursday, could generate 512x512 just fine. That's in 8G of VRAM. It worked well on the hlky branch using webui (built using gradio).

Seeing that training etc. is much more memory intensive, and wanting to get faster results, I bought an RTX 3090, which has 24G of VRAM. However it maxes out at about 1024x512, only twice as many pixels. Observing the card with GPUZ, it never actually allocates more than 13.9G.

Using the lstein branch, I can't get above 896x512. Similarly, GPUZ shows allocated VRAM never reaches 14G. The interface isn't as good as the webui on hlky either - never mind the web interface, a bigger problem is it doesn't save all the parameters alongside generated images.

This is all running using Miniconda on Windows. On Linux it may be a different story, but my gaming PC is not dual-boot (yet).

Re: Running Stable Diffusion on Your GPU with Less Than 10Gb of VRAM

#22

Post by mabbo »

reckless wrote: post #18

Would be great to be able to utilise outpainting to generate larger images in smaller tiles at full precision.

I believe I saw a repo that was doing exactly that. They also included a step at the end to reintegrate the results better.

I was also able to use the basic scripts to generate a few samples, pick one I liked, then used inpaint to expand the photo, masking out the original input so it wouldn't be altered.

Re: Running Stable Diffusion on Your GPU with Less Than 10Gb of VRAM

#25

Post by pavlov »

constantlm wrote: post #9
T0Bi wrote: post #3

Earlier quoted context omitted.

If you want to have a really good experience using stable diffusion, use this guide: https://rentry.org/GUItard - includes a nice GUI - txt2img and img2img - upscaling, face correction - many more

This is indeed a very thorough, albeit not very nicely named, guide.

Let's just pretend it's named after a background process that keeps track of your guitar.

Re: Running Stable Diffusion on Your GPU with Less Than 10Gb of VRAM

#26

Post by redox99 »

WithinReason wrote: post #16
DarthNebo wrote: post #12

Always wondered why we can't virtualize VRAM like how we did for VMs.

Good question. Bandwidth of dual channel DDR4-3600: 48 GB/s Bandwidth of PCIe 4 x16: 26 GB/s Bandiwdth of 3090 GDDR6X memory: 935.8 GB/s Since neural network evaluation is usually bandwidth limited, it's possible that pushing the data through PCI-E from CPU to GPU is actually slower than doing the evaluation on CPU only for typical neural networks. https://www.microway.com/knowledge-center-articles/performan... https…

And that's without even taking into account latency of accessing main memory through PCIe, which would make matters even worse.

Re: Running Stable Diffusion on Your GPU with Less Than 10Gb of VRAM

#28

Post by amelius »

matsemann wrote: post #20
DarthNebo wrote: post #12

Always wondered why we can't virtualize VRAM like how we did for VMs.

For training you can often divide the batch size by n (and then only apply the backprop gradient stuff after each n batches for it to be mathematically equivalent). At a cost of speed, though.

Do libraries like torch and tensorflow facilitate this?

Re: Running Stable Diffusion on Your GPU with Less Than 10Gb of VRAM

#29

Post by constantlm »

alkonaut wrote: post #27

What’s the easiest way of using SD on a Windows box? Can I run it off a Linux live USB or can it run directly under Windows? Edit: never mind this is the missing guide I had been looking for

The guide posted is for Windows 11.
Post reply on HN