From the diff, perhaps stale but:
> Carbon Emitted (Power consumption x Time x Carbon produced based on location of power grid): 11250 kg CO2 eq.
That's ... Sobering.
Running Stable Diffusion on Your GPU with Less Than 10Gb of VRAM
51–60 of 141 posts
Re: Running Stable Diffusion on Your GPU with Less Than 10Gb of VRAM
#52From the diff, perhaps stale but: > Carbon Emitted (Power consumption x Time x Carbon produced based on location of power grid): 11250 kg CO2 eq. That's ... Sobering.
My work uses a monorepo without precise dependency tracking (Bazel or similar) so every single diff builds everything and runs a ton of tests. About 6 kWh of electricity per diff. Even for typos.
Nobody seems especially bothered.
Re: Running Stable Diffusion on Your GPU with Less Than 10Gb of VRAM
#53Earlier quoted context omitted.
I feel like I'm going insane. Everyone says 512x512 should work with 8gb but when I do it I get: CUDA out of memory. Tried to allocate 3.00 GiB (GPU 0; 8.00 GiB total capacity; 5.62 GiB already allocated; 0 bytes free; 5.74 GiB reserved in total by PyTorch) any ideas? I have a 3060ti with 8gb vram... with 448x448 I get: CUDA out of memory. Tried to allocate 902.00 MiB (GPU 0; 8.00 GiB total capacity; 6.73 GiB already…
Use halfprecision float and/or the optimized forks https://github.com/basujindal/stable-diffusion https://github.com/neonsecret/stable-diffusion Or the hlky webui, that is optimized too. http://rentry.co/kretard
I've been trying to get the basujindal fork to work, but it seems to be putting all work on the CPU. I've been running the example txt2img prompt for 30 minutes now and it's still not finished. It has reserved 4Gb memory from the GPU, but the GPU doesn't appear to be doing any work, only CPU is doing work.
Re: Running Stable Diffusion on Your GPU with Less Than 10Gb of VRAM
#54It's unfortunate that this article doesn't specify the amount of VRAM needed, other than specifying it's "less than 10Gb". I have 6,1Gb of VRAM and I tried to follow the article until eventually encountering an "unable to allocate memory" error. (I'm now trying to run basujindal's repo as an alternative.)
Reduce the resolution and run with half-precision instead of full-precision and you should be able to avoid OOM errors. Author seems to have had 8GB VRAM available, so I'm guessing that's the "minimum required" for their solution.
It's not possible to halve the precision further. The precision was already dropped from float32 to float16 in the OP.
I now used parameters to drop the resolution to 256x256, and now it's running, but it's somehow broken. Every output image it produces is literally a green square.
Re: Running Stable Diffusion on Your GPU with Less Than 10Gb of VRAM
#55Earlier quoted context omitted.
I feel like I'm going insane. Everyone says 512x512 should work with 8gb but when I do it I get: CUDA out of memory. Tried to allocate 3.00 GiB (GPU 0; 8.00 GiB total capacity; 5.62 GiB already allocated; 0 bytes free; 5.74 GiB reserved in total by PyTorch) any ideas? I have a 3060ti with 8gb vram... with 448x448 I get: CUDA out of memory. Tried to allocate 902.00 MiB (GPU 0; 8.00 GiB total capacity; 6.73 GiB already…
> I feel like I'm going insane. That's the world of running machine learning models for you. Why would anything ever work the first time right? Or at least the 10th time...
Which is so silly since ML models should be the most portable thing in the world. It's just a series of math operations, not a bunch of OS/hardware specific API calls or something like that. We should be at a stage where each ML model is boiled down to a simple executable with zero dependencies at this point.
Re: Running Stable Diffusion on Your GPU with Less Than 10Gb of VRAM
#56Earlier quoted context omitted.
Good question. Bandwidth of dual channel DDR4-3600: 48 GB/s Bandwidth of PCIe 4 x16: 26 GB/s Bandiwdth of 3090 GDDR6X memory: 935.8 GB/s Since neural network evaluation is usually bandwidth limited, it's possible that pushing the data through PCI-E from CPU to GPU is actually slower than doing the evaluation on CPU only for typical neural networks. https://www.microway.com/knowledge-center-articles/performan... https…
Ok, but at least it would run.
What's the point of running it on the GPU if to do so you need to make it slower tham running in the CPU? Just run it on the CPU at that point.
Re: Running Stable Diffusion on Your GPU with Less Than 10Gb of VRAM
#57What’s the easiest way of using SD on a Windows box? Can I run it off a Linux live USB or can it run directly under Windows? Edit: never mind this is the missing guide I had been looking for
The Linux/not-Windows instructions on https://github.com/hlky/stable-diffusion/wiki/Docker-Guide worked well for me using WSL2 with nvidia-docker
Re: Running Stable Diffusion on Your GPU with Less Than 10Gb of VRAM
#58Earlier quoted context omitted.
Good question. Bandwidth of dual channel DDR4-3600: 48 GB/s Bandwidth of PCIe 4 x16: 26 GB/s Bandiwdth of 3090 GDDR6X memory: 935.8 GB/s Since neural network evaluation is usually bandwidth limited, it's possible that pushing the data through PCI-E from CPU to GPU is actually slower than doing the evaluation on CPU only for typical neural networks. https://www.microway.com/knowledge-center-articles/performan... https…
Ok, but at least it would run.
I once tried to start Firefox (back in the 2.5-3.0 days >:D) on a Celeron with 64MB RAM.
It worked perfectly fine, with the sole exception that the HDD LED was on solid the whole time, a single window took just over a literal half an hour to open, and loading a webpage took about 1-2 minutes.
But it worked.
Re: Running Stable Diffusion on Your GPU with Less Than 10Gb of VRAM
#59If you have even just 4gb stable diffusion will run fine if u go for 448x448 instead (basically the same quality).
I feel like I'm going insane. Everyone says 512x512 should work with 8gb but when I do it I get: CUDA out of memory. Tried to allocate 3.00 GiB (GPU 0; 8.00 GiB total capacity; 5.62 GiB already allocated; 0 bytes free; 5.74 GiB reserved in total by PyTorch) any ideas? I have a 3060ti with 8gb vram... with 448x448 I get: CUDA out of memory. Tried to allocate 902.00 MiB (GPU 0; 8.00 GiB total capacity; 6.73 GiB already…
Be aware python processes don't always terminate correctly when you keyboard interrupt out while using Pytorch.
Make sure you kill all python processes before restarting or some of your VRAM will be in use.
You can check with nvidia-smi how much ram is currently in use by what processes.
Re: Running Stable Diffusion on Your GPU with Less Than 10Gb of VRAM
#60Earlier quoted context omitted.
Who's paying for all the Google Collab notebooks I've been seeing around? Can I really just start and keep using it for free?
Google is paying, and yes, you can, but they will disconnect you after a while. And if you abuse it too much, you won't be able to use it until the following day... You can also buy Colab Pro and Colab Pro+, which have fewer limitations and faster GPUs.
How fast is the Colab stuff? Is Colab Pro/Pro+ a lot faster too?
I run it locally and can generate images with 50 steps in about 6 seconds per image, would it be faster for me to use Colab Free/Pro/Pro+?