Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Stable Diffusion PR optimizes VRAM, generate 576x1280 images with 6 GB VRAM

github.com

Post reply on HN
Print view View original

71–80 of 151 posts

Re: Stable Diffusion PR optimizes VRAM, generate 576x1280 images with 6 GB VRAM

#71

Post by MattRix »

p-e-w wrote: post #63
greesil wrote: post #49

Earlier quoted context omitted.

Yes, the megalomania of someone who makes their code and models freely available.

StableDiffusion isn't freely available, in the "Free Software" sense. They use the highly uncommon "CreativeML Open RAIL-M License" which is a wall of text composed of weasel words describing how the software is so incredibly advanced and dangerous that despite the authors' earnest wish to do so, they cannot in good conscience make it genuinely Free Software. These people wrote a bunch of Python code that pipes image…

Oh please, all the creators of these image AIs (OpenAI, Google, Midjourney, SD, etc) are being very very cautious with this stuff. It’s not going to end humanity, but it could easily lead to some really gross content that would paint the responsible organization in a bad light.

Re: Stable Diffusion PR optimizes VRAM, generate 576x1280 images with 6 GB VRAM

#72

Post by saurik »

I was under the impression that generating images where both dimensions were larger than 512 didn't merely require a lot of resources but didn't work well as the model was trained exclusively on 512 by 512 images and while it sort of worked ok to stretch one dimension a bit you got weird repetitions by making the overall canvas too large (as this isn't going to generate higher dpi images, merely ones with greater square inches).

Re: Stable Diffusion PR optimizes VRAM, generate 576x1280 images with 6 GB VRAM

#73

Post by JimDabell »

nl wrote: post #47

I've been using the HuggingFace diffuses repo[1] with 6GB of VRAM fine. It's well engineered, maintainable and with decent installation process. The branch in this PR[2] adds M1 Mac support with a one line patch and it runs faster than the CompVis version (1.5 iterations/sec vs 1.4 for CompVis on a 32 Gb M1 Max, I highly recommend people switching to that version for the improved flexibility. [1] https://github.com/h…

I wouldn’t recommend using that as-is. MPS doesn’t give deterministic random number generation, which means that seeds become meaningless and you won’t ever be able to reproduce something. You can work around it by generating random numbers on the CPU and then moving them to MPS, but that probably requires a fix in PyTorch.

The MPS support issue for diffusers is here:

https://github.com/huggingface/diffusers/issues/292

…and it links to the relevant PyTorch issue here:

https://github.com/pytorch/pytorch/issues/84288

Re: Stable Diffusion PR optimizes VRAM, generate 576x1280 images with 6 GB VRAM

#74

Post by IanCal »

p-e-w wrote: post #38
qdot76367 wrote: post #34

Earlier quoted context omitted.

Nah it’s just basic due diligence for releasing an open source app of this nature. Turning it off is a simple one line change, because everything is obviously named. This is not some high wall to scale. As someone that manages nsfw open source projects, this move seems fine to me. And actually kinda hilarious.

If what StableDiffusion did was ask the user something like "The prompt you entered may result in the generation of content some people find objectionable. Are you sure you want to proceed?", then I would buy your argument. As currently implemented (and the implementation took more work than a confirmation prompt would have!), it's an obvious attempt to control, rather than protect, users. They try their best to dres…

That would absolutely be more work than just subbing in an image and would neither have worked in the original implementation nor the various frontends it's been used in.

This is quite a classic HN kind of comment. Immediately assumes specific problematic intent and proposes a solution that doesn't fit the API.

Re: Stable Diffusion PR optimizes VRAM, generate 576x1280 images with 6 GB VRAM

#75

Post by the-rc »

p-e-w wrote: post #41
esjeon wrote: post #31

Earlier quoted context omitted.

Yeah, but, when it comes to generating NSFW contents, it is illegal to distribute pornographic software in some countries, and doing so can lead to domain blockage and such (i.e. South Korea, believe or not). This is something you gotta understand. Also, if the author wanted to block NSFW contents at all, I'm pretty sure one can actually make the filter inseparable from the main network. This isn't the case here AFAI…

That makes no sense. You can draw whatever you like with Photoshop. You can search for whatever you want on Google. That doesn't make the software "NSFW", and no country will block Photoshop's domain just because someone used Photoshop to create NSFW content. Don't let people who are clearly motivated by a moralistic desire to control others hide behind BS pseudo-legal excuses.

You'd be right about Photoshop, but...

https://helpx.adobe.com/photoshop/cds.html

Re: Stable Diffusion PR optimizes VRAM, generate 576x1280 images with 6 GB VRAM

#76

Post by IanCal »

p-e-w wrote: post #63
greesil wrote: post #49

Earlier quoted context omitted.

Yes, the megalomania of someone who makes their code and models freely available.

StableDiffusion isn't freely available, in the "Free Software" sense. They use the highly uncommon "CreativeML Open RAIL-M License" which is a wall of text composed of weasel words describing how the software is so incredibly advanced and dangerous that despite the authors' earnest wish to do so, they cannot in good conscience make it genuinely Free Software. These people wrote a bunch of Python code that pipes image…

If they think it's a secret weapon humanity must be prevented from using, giving it away for free with examples on how to use it with a note on the side saying "pls don't misuse thanks" seems like a very odd thing to do.

Re: Stable Diffusion PR optimizes VRAM, generate 576x1280 images with 6 GB VRAM

#77

Post by MattRix »

frognumber wrote: post #10
temp_account_32 wrote: post #6

Earlier quoted context omitted.

I am able to generate a maximum resolution of 512x768 on my 11GB 1080Ti. This seems to use almost 100% of the available RAM.

That's weird. I cap out at 512x512 on my 16GB Ampere card. Even stepping down precision doesn't help. I wonder what's different. I use it directly from Python.

You might have the n_samples (aka batch size) set to a number greater than 1? That basically multiplies the amount of VRAM you’re using.

I can generate a 512x512 on my 10gb 3080 no problem (or three 384x384 at a time)

Re: Stable Diffusion PR optimizes VRAM, generate 576x1280 images with 6 GB VRAM

#78

Post by TaylorAlexander »

In case anyone is confused by the clashing repos, here is how I was able to easily run this updated code.

Clone the original SD repo, which is what this code was built off of, and follow all the installation instructions:

https://github.com/CompVis/stable-diffusion

In that repo, replace the file ldm/modules/attention.py with this file:

https://raw.githubusercontent.com/neonsecret/stable-diffusio...

Now run a new prompt with a larger image. Note that the original model was trained on 512x512 and may lead to repetition especially if you try to increase both dimensions (this is mentioned in the SD readme) so just run with one dimension increased.

For example try the following example:

python scripts/txt2img.py --prompt "a person gardening, by claude monet" --ddim_steps 50 --seed 12000 --scale 9 --n_iter=1 --n_samples=1 --H=512 --W=1024 --skip_grid

I confirmed that if I run that command with the original attention.py, it fails due to lack of memory. With the new attention.py, it succeeds.

That said, this still uses 13GB of ram on my system.

I suppose you can check out the full repo with the updated code, which seems to have other changes, if you want to give that a try.

https://github.com/neonsecret/stable-diffusion/

I have already been using the original SD repo so I found benefit by just changing attention.py

Re: Stable Diffusion PR optimizes VRAM, generate 576x1280 images with 6 GB VRAM

#79

Post by rrobukef »

aortega wrote: post #43
qdot76367 wrote: post #11

For everyone about to comment on the garbage in the commit: It looks like the committer made their changes in the top commit, then merged the updated CompViz StableDiffusion change set on top of it for some reason. That's where the license change, rick astley image, etc come from. And yes, StableDiffusion from the original repo will rick roll you if you try to generate something that triggers its NSFW filter. Here's…

>StableDiffusion from the original repo will rick roll you if you try to generate something that triggers its NSFW filter. Just replace in scripts/txt2img.py: - x_checked_image, has_nsfw_concept = check_safety(x_samples_ddim) + x_checked_image = x_samples_ddim And be done with it.

And you save some more RAM too!

Re: Stable Diffusion PR optimizes VRAM, generate 576x1280 images with 6 GB VRAM

#80

Post by throwaway5959 »

p-e-w wrote: post #27
qdot76367 wrote: post #11

For everyone about to comment on the garbage in the commit: It looks like the committer made their changes in the top commit, then merged the updated CompViz StableDiffusion change set on top of it for some reason. That's where the license change, rick astley image, etc come from. And yes, StableDiffusion from the original repo will rick roll you if you try to generate something that triggers its NSFW filter. Here's…

> And yes, StableDiffusion from the original repo will rick roll you if you try to generate something that triggers its NSFW filter. It goes without saying that the authors of a piece of software have the right to make the software do whatever they want, but that shouldn't stop us from recognizing that AI engineers are starting to act like megalomaniac overseers who consider it part of their mission to steer humanity…

I’m sure you’re great at parties. A number of people have explained how easy this is to turn off (I did it myself in minutes without outside help by literally commenting out a line of code and changing a variable name). Getting offended at everything doesn’t solve anything and just makes half the US think we’re all snowflakes.
Post reply on HN