Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Running Stable Diffusion on Your GPU with Less Than 10Gb of VRAM

constant.meiring.nz

Post reply on HN
Print view View original

31–40 of 141 posts

Re: Running Stable Diffusion on Your GPU with Less Than 10Gb of VRAM

#31

Post by Llamamoe »

cube2222 wrote: post #4

For those without a GPU / not a powerful enough one / wanting to use SD on the go, you can start the hlky stable diffusion webui (yes, web ui) in Google Colab with this notebook[0]. It's simple and it works, using colab for processing but actually giving you a URL (ngrok-style) to open the pretty web ui in your browser. I've been using that on-the-go when not at my PC and it's been working very well for me (after try…

Who's paying for all the Google Collab notebooks I've been seeing around? Can I really just start and keep using it for free?

Re: Running Stable Diffusion on Your GPU with Less Than 10Gb of VRAM

#32

Post by mutant »

T0Bi wrote: post #3
BTCarel wrote: post #2

Can't believe how awesome these generated images are. Thank you for the guide!

If you want to have a really good experience using stable diffusion, use this guide: https://rentry.org/GUItard - includes a nice GUI - txt2img and img2img - upscaling, face correction - many more

Edgy title

Re: Running Stable Diffusion on Your GPU with Less Than 10Gb of VRAM

#33

Post by Karuma »

Llamamoe wrote: post #31
cube2222 wrote: post #4

For those without a GPU / not a powerful enough one / wanting to use SD on the go, you can start the hlky stable diffusion webui (yes, web ui) in Google Colab with this notebook[0]. It's simple and it works, using colab for processing but actually giving you a URL (ngrok-style) to open the pretty web ui in your browser. I've been using that on-the-go when not at my PC and it's been working very well for me (after try…

Who's paying for all the Google Collab notebooks I've been seeing around? Can I really just start and keep using it for free?

Google is paying, and yes, you can, but they will disconnect you after a while. And if you abuse it too much, you won't be able to use it until the following day...

You can also buy Colab Pro and Colab Pro+, which have fewer limitations and faster GPUs.

Re: Running Stable Diffusion on Your GPU with Less Than 10Gb of VRAM

#34

Post by fragmede »

Llamamoe wrote: post #31
cube2222 wrote: post #4

For those without a GPU / not a powerful enough one / wanting to use SD on the go, you can start the hlky stable diffusion webui (yes, web ui) in Google Colab with this notebook[0]. It's simple and it works, using colab for processing but actually giving you a URL (ngrok-style) to open the pretty web ui in your browser. I've been using that on-the-go when not at my PC and it's been working very well for me (after try…

Who's paying for all the Google Collab notebooks I've been seeing around? Can I really just start and keep using it for free?

Yup, totally free (with a Google account). It's run as a learning resource and there's an upsell to Collab+ and Collab Pro, but for running StableDiffusion it makes it very easy to get started!

Re: Running Stable Diffusion on Your GPU with Less Than 10Gb of VRAM

#35

Post by SuperCuber »

hwers wrote: post #30

If you have even just 4gb stable diffusion will run fine if u go for 448x448 instead (basically the same quality).

I feel like I'm going insane. Everyone says 512x512 should work with 8gb but when I do it I get:

    CUDA out of memory. Tried to allocate 3.00 GiB (GPU 0; 8.00 GiB total capacity; 5.62 GiB already allocated; 0 bytes free; 5.74 GiB reserved in total by PyTorch)
any ideas? I have a 3060ti with 8gb vram...

with 448x448 I get:

    CUDA out of memory. Tried to allocate 902.00 MiB (GPU 0; 8.00 GiB total capacity; 6.73 GiB already allocated; 0 bytes free; 6.86 GiB reserved in total by PyTorch)

Re: Running Stable Diffusion on Your GPU with Less Than 10Gb of VRAM

#36

Post by fragmede »

amelius wrote: post #28
matsemann wrote: post #20

Earlier quoted context omitted.

For training you can often divide the batch size by n (and then only apply the backprop gradient stuff after each n batches for it to be mathematically equivalent). At a cost of speed, though.

Do libraries like torch and tensorflow facilitate this?

Yes, eg https://pytorch.org/docs/stable/generated/torch.nn.parallel....

Re: Running Stable Diffusion on Your GPU with Less Than 10Gb of VRAM

#37

Post by schleck8 »

cube2222 wrote: post #4

For those without a GPU / not a powerful enough one / wanting to use SD on the go, you can start the hlky stable diffusion webui (yes, web ui) in Google Colab with this notebook[0]. It's simple and it works, using colab for processing but actually giving you a URL (ngrok-style) to open the pretty web ui in your browser. I've been using that on-the-go when not at my PC and it's been working very well for me (after try…

Also there are two optimized forks which run on https://github.com/basujindal/stable-diffusion

https://github.com/neonsecret/stable-diffusion

Re: Running Stable Diffusion on Your GPU with Less Than 10Gb of VRAM

#39

Post by schleck8 »

SuperCuber wrote: post #35
hwers wrote: post #30

If you have even just 4gb stable diffusion will run fine if u go for 448x448 instead (basically the same quality).

I feel like I'm going insane. Everyone says 512x512 should work with 8gb but when I do it I get: CUDA out of memory. Tried to allocate 3.00 GiB (GPU 0; 8.00 GiB total capacity; 5.62 GiB already allocated; 0 bytes free; 5.74 GiB reserved in total by PyTorch) any ideas? I have a 3060ti with 8gb vram... with 448x448 I get: CUDA out of memory. Tried to allocate 902.00 MiB (GPU 0; 8.00 GiB total capacity; 6.73 GiB already…

Use halfprecision float and/or the optimized forks

https://github.com/basujindal/stable-diffusion

https://github.com/neonsecret/stable-diffusion

Or the hlky webui, that is optimized too.

http://rentry.co/kretard

Re: Running Stable Diffusion on Your GPU with Less Than 10Gb of VRAM

#40

Post by jeroenhd »

SuperCuber wrote: post #35
hwers wrote: post #30

If you have even just 4gb stable diffusion will run fine if u go for 448x448 instead (basically the same quality).

I feel like I'm going insane. Everyone says 512x512 should work with 8gb but when I do it I get: CUDA out of memory. Tried to allocate 3.00 GiB (GPU 0; 8.00 GiB total capacity; 5.62 GiB already allocated; 0 bytes free; 5.74 GiB reserved in total by PyTorch) any ideas? I have a 3060ti with 8gb vram... with 448x448 I get: CUDA out of memory. Tried to allocate 902.00 MiB (GPU 0; 8.00 GiB total capacity; 6.73 GiB already…

Others may have reduced the batch size (n_samples) to reduce the memory load. A lower batch size will significantly help with the memory consumption.

This comment: https://news.ycombinator.com/item?id=32710550 talks about running SD with 8GiB of VRAM and mentions needing to reduce this parameter to 1 to get it to output right.

Post reply on HN