Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Running Stable Diffusion on Your GPU with Less Than 10Gb of VRAM

constant.meiring.nz

Post reply on HN
Print view View original

111–120 of 141 posts

Re: Running Stable Diffusion on Your GPU with Less Than 10Gb of VRAM

#111

Post by coolspot »

layer8 wrote: post #102

I’ll get downvoted, but it’s a genuine question: Will “a photo of tits and ass” generate photos of birds with donkeys, or will it rickroll you [0]? [0] https://twitter.com/qDot/status/1565076751465648128

Just tested it locally[0] with two prompts (all default params): "A photo of tits and ass" and "A photo of tits (birds) and ass (donkey)"

Result: https://imgur.com/a/c1GM28U (NSFW)

Censored version would just replace anything NSFW with a picture of Rick Astley (for real [1]).

[0] - https://github.com/hlky/stable-diffusion

[1] - https://github.com/CompVis/stable-diffusion/issues/120

Re: Running Stable Diffusion on Your GPU with Less Than 10Gb of VRAM

#112

Post by skybrian »

davidy123 wrote: post #105

Not an Apple guy, but I think an Apple M chip will run at ⅓ the speed of a top end RTX GPU, however it uses system memory, so it can easily be 32GB or 64Gb. That's pretty compelling, and if this is really a new class of application, NVidia is going to have to think about more memory for mainstream-ish products.

This is a specialty application. I don't think it's going to be big enough to drive consumer technology like gaming?

Particularly since cloud services are likely to be competitive and work for anyone.

Re: Running Stable Diffusion on Your GPU with Less Than 10Gb of VRAM

#113

Post by layer8 »

coolspot wrote: post #111
layer8 wrote: post #102

I’ll get downvoted, but it’s a genuine question: Will “a photo of tits and ass” generate photos of birds with donkeys, or will it rickroll you [0]? [0] https://twitter.com/qDot/status/1565076751465648128

Just tested it locally[0] with two prompts (all default params): "A photo of tits and ass" and "A photo of tits (birds) and ass (donkey)" Result: https://imgur.com/a/c1GM28U (NSFW) Censored version would just replace anything NSFW with a picture of Rick Astley (for real [1]). [0] - https://github.com/hlky/stable-diffusion [1] - https://github.com/CompVis/stable-diffusion/issues/120

Thanks for actually trying that out. Surprisingly few tits with those asses. The (animal) ass-tit chimeras are amusing.

Re: Running Stable Diffusion on Your GPU with Less Than 10Gb of VRAM

#114

Post by coolspot »

davidy123 wrote: post #105

Not an Apple guy, but I think an Apple M chip will run at ⅓ the speed of a top end RTX GPU, however it uses system memory, so it can easily be 32GB or 64Gb. That's pretty compelling, and if this is really a new class of application, NVidia is going to have to think about more memory for mainstream-ish products.

It is 50x times slower on M1 than on RTX 3090.

M1 takes ~4.2s per iteration, 3.5 minutes per image [0].

RTX 3090 takes ~4.7s per image (all 50 iterations) [1].

[0] - https://wandb.ai/morgan/stable-diffusion/reports/Running-Sta...

[1] - trust me bro

Re: Running Stable Diffusion on Your GPU with Less Than 10Gb of VRAM

#115

Post by shrimpx »

coolspot wrote: post #114
davidy123 wrote: post #105

Not an Apple guy, but I think an Apple M chip will run at ⅓ the speed of a top end RTX GPU, however it uses system memory, so it can easily be 32GB or 64Gb. That's pretty compelling, and if this is really a new class of application, NVidia is going to have to think about more memory for mainstream-ish products.

It is 50x times slower on M1 than on RTX 3090. M1 takes ~4.2s per iteration, 3.5 minutes per image [0]. RTX 3090 takes ~4.7s per image (all 50 iterations) [1]. [0] - https://wandb.ai/morgan/stable-diffusion/reports/Running-Sta... [1] - trust me bro

Btw that's the kind of perf I see on my M1, but I keep seeing "0.00G VRAM used" for each generation. I wonder what that's about. In Activity Monitor I do see the GPU being used.

Re: Running Stable Diffusion on Your GPU with Less Than 10Gb of VRAM

#116

Post by davidy123 »

coolspot wrote: post #114
davidy123 wrote: post #105

Not an Apple guy, but I think an Apple M chip will run at ⅓ the speed of a top end RTX GPU, however it uses system memory, so it can easily be 32GB or 64Gb. That's pretty compelling, and if this is really a new class of application, NVidia is going to have to think about more memory for mainstream-ish products.

It is 50x times slower on M1 than on RTX 3090. M1 takes ~4.2s per iteration, 3.5 minutes per image [0]. RTX 3090 takes ~4.7s per image (all 50 iterations) [1]. [0] - https://wandb.ai/morgan/stable-diffusion/reports/Running-Sta... [1] - trust me bro

OK, I must have misread some comments. Thanks for the update.

Re: Running Stable Diffusion on Your GPU with Less Than 10Gb of VRAM

#118

Post by glenneroo »

ladberg wrote: post #72
barrkel wrote: post #21

I started out using my old GTX 1080 on Thursday, could generate 512x512 just fine. That's in 8G of VRAM. It worked well on the hlky branch using webui (built using gradio). Seeing that training etc. is much more memory intensive, and wanting to get faster results, I bought an RTX 3090, which has 24G of VRAM. However it maxes out at about 1024x512, only twice as many pixels. Observing the card with GPUZ, it never actu…

Just checking, are you also using the same GPU for rendering your desktop? If so then try switching over to your integrated GPU or the 1080 if it’s still attached so you can leave 100% of the 3090 available to the network.

That probably won't help much since OP said they aren't even using 14GB of VRAM. I have dual GPUs and use the 2nd one (3060 Ti with no monitors connected) for rendering, which is nice because I have the full 8GB free.

Re: Running Stable Diffusion on Your GPU with Less Than 10Gb of VRAM

#119

Post by glenneroo »

cbozeman wrote: post #83
tbalsam wrote: post #73

Earlier quoted context omitted.

Recommendations: - Linux w/ Display drivers on CPU (and just...ditch miniconda please) - Use lower FP precision mode if available to use the tensor cores (also to double "effective" memory) - Batch things! - I don't know what the max resolution of the diffusion network is, you may have to just simply tile it past a certain point (with overlap please! ;P Hope that helps somewhat. A 3090 should be more than enough for…

I ordered an NVIDIA Tesla K80 off eBay (and the power adapter... and the blower fan shroud, etc.) and intend to install it when it arrives around Thursday or Friday. I'm hoping that after I install the NVIDIA Linux datacenter drivers I'll be able to use the card with SD. My only worry is that because the K80 is two GPUs on one board, that it might only utilize one of them, with only 12 GB of VRAM instead of both chip…

You'll be able to render but it won't be fast, those CUDA cores are ancient and VRAM speed is slow. Check the Stable Diffusion discord for more info, but I found these comments:

> one minute per 512x512 @ 50 steps

> 1m20s to run 50 ddims on 512x512 vs 2080 ti in 12 seconds

You'll have to run the optimized model as well, since you can't connect the 2x 12GB together.

Re: Running Stable Diffusion on Your GPU with Less Than 10Gb of VRAM

#120

Post by glenneroo »

SuperCuber wrote: post #35
hwers wrote: post #30

If you have even just 4gb stable diffusion will run fine if u go for 448x448 instead (basically the same quality).

I feel like I'm going insane. Everyone says 512x512 should work with 8gb but when I do it I get: CUDA out of memory. Tried to allocate 3.00 GiB (GPU 0; 8.00 GiB total capacity; 5.62 GiB already allocated; 0 bytes free; 5.74 GiB reserved in total by PyTorch) any ideas? I have a 3060ti with 8gb vram... with 448x448 I get: CUDA out of memory. Tried to allocate 902.00 MiB (GPU 0; 8.00 GiB total capacity; 6.73 GiB already…

I've been running it fine on my 3060 Ti, then again I don't have any monitors connected so the full 8GB is free. Check VRAM usage, I'm guessing you don't have 8GB free, more like 5-6GB, since you have monitors connected.

Also, you could try Visions of Chaos and use the Mode > Machine Learning > Text-to-Image > Stable Diffusion. It also has tons of other AI tools e.g. image-to-text captioning, diffusion model training, mandelbrot, music, and a ton more. The dev(s) push out updates almost every day.

Warning: You will first need to go through the 12 steps of Machine Learning setup first[0], then it will download 3-400GB of models since it has scripts for pretty much every latent diffusion out there, some of which e.g. Disco Diffusion I find to still give more interesting results and you can get much higher res on a 3060 Ti, plus you have a TON more parameters to play with, not to mention you can train your own models and load those in (which I've been doing the past few weeks using my photography to get away from using unlicensed imagery :)

[0] https://softology.pro/tutorials/tensorflow/tensorflow.htm

Post reply on HN