Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Running Stable Diffusion on Your GPU with Less Than 10Gb of VRAM

constant.meiring.nz

Post reply on HN
Print view View original

131–140 of 141 posts

Re: Running Stable Diffusion on Your GPU with Less Than 10Gb of VRAM

#131

Post by r2_pilot »

spapas82 wrote: post #99

I'd like to confirm that this works in my GTX 2060 with 6 GB VRAM on windows. I didn't do any modifications on the provided source code; faces are a little problematic. I don't use anaconda so I created a new venv with python 3.10, installed the requirements as proposed, registered with hugging face and create the api key and run the provided source code. Any way to improve the quality of the faces? Also how could I…

It's my understanding that v1.5 will be coming out in a few weeks; I recall that hands and faces will be better-trained in the new model. I'm about to try what you did (install requirements manually) to get it to run in PyCharm on Windows. Neither miniconda nor anaconda really worked for me after spending time trying to get it to pick up the dependencies.

Re: Running Stable Diffusion on Your GPU with Less Than 10Gb of VRAM

#132

Post by [deleted] »

baobabKoodaa wrote: post #71
prettydeep wrote: post #68

Earlier quoted context omitted.

Use the original SD repo. But modify the txt2img.py according to: https://github.com/CompVis/stable-diffusion/issues/86#issuec...

I now did everything I could to constrain the memory usage of the original SD repo, I was finally able to get it to run, and it produced green squares as output :( What I did: - scripts/txt2img.py, function - load_model_from_config, line - 63, change from: model.cuda() to model.cuda().half() - removed invisible watermarking - reduced n_samples to 1 - reduced resolution to 256x256 - removed sfw filter Just can't get i…

[deleted]

Re: Running Stable Diffusion on Your GPU with Less Than 10Gb of VRAM

#133

Post by throwaway675309 »

fortyseven wrote: post #128
hackernewds wrote: post #19

Earlier quoted context omitted.

> --ULTIMATE GUI RETARD GUIDE--

Do we REALLY this kind of garbage associated with SD? Bad enough I see trashy right-wing extremist shit over on the Stable Diffusion discord server zip past now and then. I'll pass on that guide. Hopefully they grow the fuck up at some point.

Unfortunately as long as there are people who are easily triggered by this sort of thing (seems like they got a bit of a rise out of you) they'll continue in this fashion.

Re: Running Stable Diffusion on Your GPU with Less Than 10Gb of VRAM

#134

Post by cbozeman »

glenneroo wrote: post #119
cbozeman wrote: post #83

Earlier quoted context omitted.

I ordered an NVIDIA Tesla K80 off eBay (and the power adapter... and the blower fan shroud, etc.) and intend to install it when it arrives around Thursday or Friday. I'm hoping that after I install the NVIDIA Linux datacenter drivers I'll be able to use the card with SD. My only worry is that because the K80 is two GPUs on one board, that it might only utilize one of them, with only 12 GB of VRAM instead of both chip…

You'll be able to render but it won't be fast, those CUDA cores are ancient and VRAM speed is slow. Check the Stable Diffusion discord for more info, but I found these comments: > one minute per 512x512 @ 50 steps > 1m20s to run 50 ddims on 512x512 vs 2080 ti in 12 seconds You'll have to run the optimized model as well, since you can't connect the 2x 12GB together.

> since you can't connect the 2x 12GB together.

Any idea why this is?

Re: Running Stable Diffusion on Your GPU with Less Than 10Gb of VRAM

#135

Post by cdelsolar »

moron4hire wrote: post #107
cdelsolar wrote: post #101

anyone know how to get conda running on arch linux? `conda init bash` gives me some Python errors.

Yeah, I don't think that's an Arch Linux problem. I had similar problems on Windows, and one version of the project even was supposedly setup to run in Docker. What is the point of setting up Docker if the whole setup and build process is not turnkey? Seems like all of these projects are broken until you speak shibboleth by guessing at random python incantations. By this point, it's starting to feel intentional, like…

I'm on Python 3.10, so I probably have to wait till they fix something.

Re: Running Stable Diffusion on Your GPU with Less Than 10Gb of VRAM

#136

Post by nickthegreek »

hombre_fatal wrote: post #103

I've been running Stable Diffusion on my M1 Macbook since the thread a few days ago about doing just that. I am comically bad at getting it to generate what I want. e.g. "A furry watermelon" or "A dog flexing its biceps" just generates normal watermelons and normal dogs most of the time. Any tips?

Use www.lexica.art for prompts that work, modify the keywords for your needs.

Re: Running Stable Diffusion on Your GPU with Less Than 10Gb of VRAM

#137

Post by moron4hire »

cdelsolar wrote: post #135
moron4hire wrote: post #107

Earlier quoted context omitted.

Yeah, I don't think that's an Arch Linux problem. I had similar problems on Windows, and one version of the project even was supposedly setup to run in Docker. What is the point of setting up Docker if the whole setup and build process is not turnkey? Seems like all of these projects are broken until you speak shibboleth by guessing at random python incantations. By this point, it's starting to feel intentional, like…

I'm on Python 3.10, so I probably have to wait till they fix something.

Yeah, I don't really get why Python developers tolerate this state of being where minor point releases are so commonly incompatible with each other, or having no standard way to manage building against different versions. I also don't get why ML developers continue with Python, considering these issues.

But, that's the point of the Miniconda dependency. By using Conda, the project can be setup locally with the Python version it expects without clobbering your local, system-level install.

It was still a bit of a pain to learn how to use Conda, but it worked out a little better than figuring it out on my own.

Re: Running Stable Diffusion on Your GPU with Less Than 10Gb of VRAM

#138

Post by simcop2387 »

cbozeman wrote: post #134
glenneroo wrote: post #119

Earlier quoted context omitted.

You'll be able to render but it won't be fast, those CUDA cores are ancient and VRAM speed is slow. Check the Stable Diffusion discord for more info, but I found these comments: > one minute per 512x512 @ 50 steps > 1m20s to run 50 ddims on 512x512 vs 2080 ti in 12 seconds You'll have to run the optimized model as well, since you can't connect the 2x 12GB together.

> since you can't connect the 2x 12GB together. Any idea why this is?

The architecture of the k80 just doesn't allow memory sharing or pooling. The newer systems (Pascal and above I think?) can allow it on the right hardware but it's all datacenter and workstation cards for that.

Re: Running Stable Diffusion on Your GPU with Less Than 10Gb of VRAM

#139

Post by cbozeman »

simcop2387 wrote: post #138
cbozeman wrote: post #134

Earlier quoted context omitted.

> since you can't connect the 2x 12GB together. Any idea why this is?

The architecture of the k80 just doesn't allow memory sharing or pooling. The newer systems (Pascal and above I think?) can allow it on the right hardware but it's all datacenter and workstation cards for that.

Man, feel like I shoulda picked up the M40 instead then, since it was 24 GB VRAM in total. :(

Oh well, I've spent $90 on dinners and had far less fun than I will have with this video card when it arrives, so I can't say it was wasted money... and I can always just buy an M40 off eBay.

If RTX 3090 prices keep dropping through the floor, I may just bite the bullet and pick one up. I saw a ZOTAC on sale for $999 recently, which is $500 less than the launch MSRP of $1499 (which honestly is where it should have launched anyway... so far as I'm concerned, these cards only just now hit reasonable pricing).

Re: Running Stable Diffusion on Your GPU with Less Than 10Gb of VRAM

#140

Post by nilolo »

XorNot wrote: post #67

I have this running on my fairly mundane Radeon 5600XT at about 1 minute per image generated (under rootless podman, which is the real cool news to me) which isn't bad all things considered. Definitely get some interesting sounds from coil whine when it's going.

Could you share which repo you got running? I followed this guide (https://gitgudblog.vercel.app/posts/stable-diffusion-amd-win...) that uses Onnx to get it running on my 5700XT but I'm not happy with the performance at about 2m30s per image.
Post reply on HN