Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Stable Diffusion PR optimizes VRAM, generate 576x1280 images with 6 GB VRAM

github.com

Post reply on HN
Print view View original

121–130 of 151 posts

Re: Stable Diffusion PR optimizes VRAM, generate 576x1280 images with 6 GB VRAM

#121

Post by mobiclick »

bawolff wrote: post #93
Georgelemental wrote: post #57

Earlier quoted context omitted.

The proverb describes a state of mind one should aspire to. "Words will never hurt me" not because saying mean things can never do harm (it obviously can), but because I make an effort to control my emotions and reactions. Just as physical health is improved by a nutritious diet and regular exercise, mental health is improved by proper habits of mind.

Refusing to acknowledge the affect of hurtful words on yourself, is pretty much the opposite of "proper habits of the mind". Denial does not make healthy people.

It's not denial so much as it is not caring to the point where you get upset. I think that's the strength of mind that GP is referring to.

Re: Stable Diffusion PR optimizes VRAM, generate 576x1280 images with 6 GB VRAM

#122

Post by archi42 »

oefrha wrote: post #26
piyh wrote: post #19

Earlier quoted context omitted.

As a learning opportunity for people like me, what does a good PR look like for a large change?

I would say large contributions from non-members generally work rather terribly in open source. If you absolutely have to, - Communicate ahead of time; don't surprise maintainers with sweeping architectural changes or huge features no one wants or would like to review; - Try to break up changes into logical units that can be understood and reviewed independently; - Write useful and detailed commit messages (some bad…

I had good success with this flow for major changes (both in-house as well as contributing to FOSS). The target was clear & discussed upfront (sans details that pop up during the actual work), and achieved by a series of consecutive PRs. It's important to be constructive, communicate the intent properly and give the other party enough time to digest the info and think properly about it. Obviously this also depends on the maintainers and their mentality.

Re: Stable Diffusion PR optimizes VRAM, generate 576x1280 images with 6 GB VRAM

#123

Post by JimDabell »

nl wrote: post #120
JimDabell wrote: post #73

Earlier quoted context omitted.

I wouldn’t recommend using that as-is. MPS doesn’t give deterministic random number generation, which means that seeds become meaningless and you won’t ever be able to reproduce something. You can work around it by generating random numbers on the CPU and then moving them to MPS, but that probably requires a fix in PyTorch. The MPS support issue for diffusers is here: https://github.com/huggingface/diffusers/issues/2…

That's really annoying, even though I hadn't noticed it until now. I can confirm this is an issue. But the situation seems to be the same on the CompVis derived repos, right? So this is no worse off, but with better engineered and faster code.

Yeah, but with the CompVis derived repos, it’s pretty easy to go in and change all the calls to PyTorch random number generators.

Having said that, the last comment [0] on the PyTorch issue gave me the idea of monkey patching the random functions. The supplied code assumes you’re always passing in a generator, which is not true in this case, but if you monkey patch the three rand/randn/randn_like functions to do nothing but swap out the device parameter for 'cpu' and then call to('ops') on the return value, it’s enough to get stable seed functionality for the CompVis derived repos without modifying their code, so I’m guessing it will probably work for diffusers as well.

Also, it’s probably a bug in the CompVis code, but even after you fix the random number generator, the very first run in a session uses an incorrect seed. The workaround is to generate an image once to throw away whenever you start a new session.

[0] https://github.com/pytorch/pytorch/issues/84288#issuecomment...

Re: Stable Diffusion PR optimizes VRAM, generate 576x1280 images with 6 GB VRAM

#124

Post by liuliu »

WithinReason wrote: post #111

The diff that apparently does the RAM optimisation, quite simple and something to learn form: https://github.com/basujindal/stable-diffusion/commit/47f878...

Yeah, stable diffusion's PyTorch code is not optimized for inference memory usage at start. I am looking at the code now and it seems that if it is converted to static graph, there are probably a bit more opportunities (I only looked at CLIP model and UNet model it uses today, not sure about the Autoencoder yet).

Re: Stable Diffusion PR optimizes VRAM, generate 576x1280 images with 6 GB VRAM

#125

Post by IanCal »

macrolime wrote: post #81
IanCal wrote: post #74

Earlier quoted context omitted.

That would absolutely be more work than just subbing in an image and would neither have worked in the original implementation nor the various frontends it's been used in. This is quite a classic HN kind of comment. Immediately assumes specific problematic intent and proposes a solution that doesn't fit the API.

Now instead users have to spend hours upon hours editing docker images to remove it. Much easier.

That's neither true not would it have been helped by making the original code require input during generation.

They put a simple check in, that tries to avoid returning nsfw images so that you don't get that back despite a more 'innocent' prompt. It's trivial to remove and is only part of the demonstration scripts, it is not part of the model or anything fundamental to it's workings.

Re: Stable Diffusion PR optimizes VRAM, generate 576x1280 images with 6 GB VRAM

#126

Post by IanCal »

bawolff wrote: post #92
Kiro wrote: post #88

Earlier quoted context omitted.

You have no point whatsoever. You're fighting windmills.

I disagree that the rick roll is inappropriate, but i also hate the argument that just because its open source, the programmers are above criticism. I think its reasonable people can dislike or even morally object to the decisions open source maintainers make (even if i dont particularly object to this specific instance)

They're not above criticism, but what they've done is a very small layer that's easily removed. You can do it, and release a version without it, in minutes. You can do it in less time than it takes to write a long comment decrying it.

It's not fundamental to SD in any way, and their suggestion wouldn't have worked.

Re: Stable Diffusion PR optimizes VRAM, generate 576x1280 images with 6 GB VRAM

#127

Post by IanCal »

wolfd wrote: post #98
p-e-w wrote: post #38

Earlier quoted context omitted.

If what StableDiffusion did was ask the user something like "The prompt you entered may result in the generation of content some people find objectionable. Are you sure you want to proceed?", then I would buy your argument. As currently implemented (and the implementation took more work than a confirmation prompt would have!), it's an obvious attempt to control, rather than protect, users. They try their best to dres…

The implementation as it stands is actually easier the way they do it. They replace the image by scaling Rick to the image size. That means that they can still automatically put your output images in a grid without any extra code. They could have zeroed the pixels, but that could have led to confusion about if things were working or not. It's silly and you're able to patch it out if you so desire. But most importantl…

Absolutely. The limit isn't in what you ask for, it's in what's returned. The whole point is to try and stop the user being shown nsfw images despite sfw prompts, and since you can generate multiple results per prompt adding an interactive check on each one doesn't make sense.

Re: Stable Diffusion PR optimizes VRAM, generate 576x1280 images with 6 GB VRAM

#128

Post by Werewolf255 »

p-e-w wrote: post #38
qdot76367 wrote: post #34

Earlier quoted context omitted.

Nah it’s just basic due diligence for releasing an open source app of this nature. Turning it off is a simple one line change, because everything is obviously named. This is not some high wall to scale. As someone that manages nsfw open source projects, this move seems fine to me. And actually kinda hilarious.

If what StableDiffusion did was ask the user something like "The prompt you entered may result in the generation of content some people find objectionable. Are you sure you want to proceed?", then I would buy your argument. As currently implemented (and the implementation took more work than a confirmation prompt would have!), it's an obvious attempt to control, rather than protect, users. They try their best to dres…

You know you can download the code yourself, right? You need not be reliant on their web interface if you're having these control issues.

It's also clearly in place because they don't have an age disclaimer, which would likely open them up to some sort of liability (or additional liability). Again, it's open source, and freely available, so like. . . this feels like a you problem, sorry.

Re: Stable Diffusion PR optimizes VRAM, generate 576x1280 images with 6 GB VRAM

#129

Post by wholinator2 »

mobiclick wrote: post #121
bawolff wrote: post #93

Earlier quoted context omitted.

Refusing to acknowledge the affect of hurtful words on yourself, is pretty much the opposite of "proper habits of the mind". Denial does not make healthy people.

It's not denial so much as it is not caring to the point where you get upset. I think that's the strength of mind that GP is referring to.

I agree. Just because we have more advanced diagnosis and detection tools these days doesn't mean everything has to be a problem. The internet has really shaken up social norms and signaling and the kids are the first ones to enter the new world. I believe once we have a generation of fully formed 30-50 year olds who grew up with this strange social media, then we'll have the proper understandings and social knowledge to handle it well. Kinda like how parents these days can attempt to relate to their kids in high school. We need the elder wisdom there.

But I do believe that right now we're kinda off track. We almost venerate the act of being hurt. Everyone likes attention and nothing gets such protection by certain classes as having been offended or wronged by some other class. Social signals are currently built to display virtue and so people will go out of their way to display their support of the wronged. I _do_ believe that this is the correct direction to move from where we were, but I think it's gone a little too far and needs time to rebound.

Being a victim is the fastest way to go from zero to hero (reach millions of people) these days and it's also seemingly the least likely way to backfire. People are much more hesitant to bring up the wrongs of someone who's currently being defended for fear of ending up being placed in the out group and ostracized from the signaling group.

Re: Stable Diffusion PR optimizes VRAM, generate 576x1280 images with 6 GB VRAM

#130

Post by wholinator2 »

wholinator2 wrote: post #129
mobiclick wrote: post #121

Earlier quoted context omitted.

It's not denial so much as it is not caring to the point where you get upset. I think that's the strength of mind that GP is referring to.

I agree. Just because we have more advanced diagnosis and detection tools these days doesn't mean everything has to be a problem. The internet has really shaken up social norms and signaling and the kids are the first ones to enter the new world. I believe once we have a generation of fully formed 30-50 year olds who grew up with this strange social media, then we'll have the proper understandings and social knowledg…

This all contributes to a social custom of looking to be wronged, so that you may point out the evil of the person/generation/world. However, people aren't idiots and can _sometimes_ tell if you're just faking for clout. This leads to a feedback loop of needing to be really hurt, seeing other people be really hurt, believing your really hurt, and finally internalizing that pain and trauma. Things do hurt, bad words are called bad for a reason. Society might be better if we were all nice. But every bad word and every microaggression does not need to become such a large roadblock to personal freedom. People are chaining themselves to the road with this stuff. Dieing on hills that require them to have been personally wronged, using their own pain as a way to shut down criticism. Yes, people hurt, and things can be bad. But it's also entirely possible to see something hurtful and continue life without it hurting you. It's 100% doable to actually not be hurt, not just ignore it, but to construct a self esteem and understanding that allows you to not be shanked by every half difficult social interaction that occurs.

Mental health awareness is good. But social signals lead individuals to believing they must be hurt to be a part of the in group. Virtue through suffering is an incredibly effective signaling mechanism

Post reply on HN