Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Running Stable Diffusion on Your GPU with Less Than 10Gb of VRAM

constant.meiring.nz

Post reply on HN
Print view View original

121–130 of 141 posts

Re: Running Stable Diffusion on Your GPU with Less Than 10Gb of VRAM

#121

Post by coolspot »

shrimpx wrote: post #115
coolspot wrote: post #114

Earlier quoted context omitted.

It is 50x times slower on M1 than on RTX 3090. M1 takes ~4.2s per iteration, 3.5 minutes per image [0]. RTX 3090 takes ~4.7s per image (all 50 iterations) [1]. [0] - https://wandb.ai/morgan/stable-diffusion/reports/Running-Sta... [1] - trust me bro

Btw that's the kind of perf I see on my M1, but I keep seeing "0.00G VRAM used" for each generation. I wonder what that's about. In Activity Monitor I do see the GPU being used.

SD measures VRAM usage by calling a specific pyTorch method which usually wraps CUDA call.

I guess whomever ported that to M1 just haven’t implemented that method.

Re: Running Stable Diffusion on Your GPU with Less Than 10Gb of VRAM

#122

Post by barrkel »

glenneroo wrote: post #118
ladberg wrote: post #72

Earlier quoted context omitted.

Just checking, are you also using the same GPU for rendering your desktop? If so then try switching over to your integrated GPU or the 1080 if it’s still attached so you can leave 100% of the 3090 available to the network.

That probably won't help much since OP said they aren't even using 14GB of VRAM. I have dual GPUs and use the 2nd one (3060 Ti with no monitors connected) for rendering, which is nice because I have the full 8GB free.

I was looking at GPU graphs and neglected my physical RAM. This machine only has 32G and I didn't notice I was hitting a ceiling on memory allocations too - I took the error message about GPU memory allocations at face value.

I bumped my system commit cap (increased paging file size) by 24G and now I can use all my VRAM.

Re: Running Stable Diffusion on Your GPU with Less Than 10Gb of VRAM

#123

Post by barrkel »

MattRix wrote: post #64
barrkel wrote: post #21

I started out using my old GTX 1080 on Thursday, could generate 512x512 just fine. That's in 8G of VRAM. It worked well on the hlky branch using webui (built using gradio). Seeing that training etc. is much more memory intensive, and wanting to get faster results, I bought an RTX 3090, which has 24G of VRAM. However it maxes out at about 1024x512, only twice as many pixels. Observing the card with GPUZ, it never actu…

Have you tried something like 1024x768? Going to full 1024x1024 would double your VRAM usage so I can see why that wouldn’t work. For my uses, the real benefit of having more VRAM is that you can generate more images simultaneously. My 3080 can generate only one 512x512 in 7 seconds but three 384x384 in that same timeframe. It’s allowed me to generate grids of hundreds of images in just a few minutes.

1024x768 didn't work until I bumped my system commit cap (i.e. increased my paging file max size). I hadn't paid attention to system memory, this box only has 32G.

Re: Running Stable Diffusion on Your GPU with Less Than 10Gb of VRAM

#124

Post by neurostimulant »

baobabKoodaa wrote: post #54
capableweb wrote: post #49

Earlier quoted context omitted.

Reduce the resolution and run with half-precision instead of full-precision and you should be able to avoid OOM errors. Author seems to have had 8GB VRAM available, so I'm guessing that's the "minimum required" for their solution.

It's not possible to halve the precision further. The precision was already dropped from float32 to float16 in the OP. I now used parameters to drop the resolution to 256x256, and now it's running, but it's somehow broken. Every output image it produces is literally a green square.

I also got green square when running with half precision on gtx 1650. Try 384x384 with full precision instead, or use the intel cpu fork.

Re: Running Stable Diffusion on Your GPU with Less Than 10Gb of VRAM

#125

Post by neurostimulant »

cdelsolar wrote: post #101

anyone know how to get conda running on arch linux? `conda init bash` gives me some Python errors.

Pretty easy actually. Just install miniconda from here [1]. It'll add some codes into your bashrc / zshrc so you'll need to reopen your terminal after installation.

[1] https://docs.conda.io/en/latest/miniconda.html#linux-install...

Re: Running Stable Diffusion on Your GPU with Less Than 10Gb of VRAM

#127

Post by XorNot »

baobabKoodaa wrote: post #91
XorNot wrote: post #89

Earlier quoted context omitted.

You need to be in full precision mode in that case. Running on my AMD card this was necessary.

Runs out of memory in full precision mode.

Try the basujindal fork https://github.com/basujindal/stable-diffusion - this is what works for me on 8gb of VRAM (without maybe 6.5 available from whatever my DE is using up).

On Linux with AMD `radeontop` is quite informative.

Re: Running Stable Diffusion on Your GPU with Less Than 10Gb of VRAM

#128

Post by fortyseven »

hackernewds wrote: post #19
sdflhasjd wrote: post #10

Earlier quoted context omitted.

It originates from 4chan /vg/ & /g/ boards

> --ULTIMATE GUI RETARD GUIDE--

Do we REALLY this kind of garbage associated with SD?

Bad enough I see trashy right-wing extremist shit over on the Stable Diffusion discord server zip past now and then.

I'll pass on that guide. Hopefully they grow the fuck up at some point.

Re: Running Stable Diffusion on Your GPU with Less Than 10Gb of VRAM

#130

Post by r2_pilot »

yoru-sulfur wrote: post #76
tbalsam wrote: post #73

Earlier quoted context omitted.

Recommendations: - Linux w/ Display drivers on CPU (and just...ditch miniconda please) - Use lower FP precision mode if available to use the tensor cores (also to double "effective" memory) - Batch things! - I don't know what the max resolution of the diffusion network is, you may have to just simply tile it past a certain point (with overlap please! ;P Hope that helps somewhat. A 3090 should be more than enough for…

> and just...ditch miniconda please Why the recommendation to stop using miniconda?

Well, for starters(not that proficient with PyCharm as I only use it for less-professional use), I have tried for a few days to get PyCharm to use the miniconda environment on Windows, to no avail. It at least works on the command line so I get to play around with it. I'm going to spend more time tomorrow trying alternatives.
Post reply on HN