Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Running Stable Diffusion XL 1.0 in 298MB of RAM

github.com

Post reply on HN
Print view View original

141–150 of 166 posts

Re: Running Stable Diffusion XL 1.0 in 298MB of RAM

#141

Post by ionwake »

isoprophlex wrote: post #56
chasd00 wrote: post #49

Earlier quoted context omitted.

that's an awesome idea, do you have a link to more information?

Ill write it up once it's done and post here, if it gains traction you might see it haha. In all seriousness I can give a brief overview: - I'll probably offload the image generation to the 5 year old intel nuc I already have as a home automation server, comfyUI in CPU mode takes 20-30 mins for a generation. Ideally it's all self contained on the Pi but that might be beyond me, skill wise. - prompts are composed by t…

Sounds awesome what’s your Twitter ?

Re: Running Stable Diffusion XL 1.0 in 298MB of RAM

#142

Post by xcdzvyn »

yowlingcat wrote: post #122
pphysch wrote: post #65

Earlier quoted context omitted.

That is virtually impossible because Turing-complete systems are everywhere

I wonder if there's an analogy to be made here to DRM. In theory, yes, DRM shouldn't be possible, but in practice, manufacturers have been able to hobble hardware acceleration behind trusted computing model. Often, they do a poor job and it gets cracked (as with HDCP [1], and UWP [2]). The question in my head is whether the failures in their approaches are due to a flaw in the implementation (in which case it's pract…

Yeah I think it's fair to assume DRM will be a never-ending cat and mouse between developers and end-users.

Netflix for example can implement any DRM tech they want -- ultimately they're putting a picture on my screen, and it's impossible to stop me from extracting it.

Re: Running Stable Diffusion XL 1.0 in 298MB of RAM

#143

Post by xcdzvyn »

phh wrote: post #19
samr71 wrote: post #9

So this should be it for trying to regulate stable diffusion type tech, right? If these models and their inference infra can be shrunk down to be runnable on a PS2, it doesn't seem like it's possible to stop this tech without a totalitarian surveillance state (and barely even then!).

So this should be it for trying to regulate theft, right? If you can open a window without any tool other than your own body. It doesn't seem like it's possible to stop thefts without a totalitarian surveillance state (and barely event then!). Or same can be said about media "piracy". Or ransomwares. States have forever regulated things that are not possible to enforce purely technically.

Yes, it is impossible to stop theft.

Re: Running Stable Diffusion XL 1.0 in 298MB of RAM

#145

Post by vitoplantamura »

SigmundurM wrote: post #7
omneity wrote: post #3

Fascinating. The money quote: "OnnxStream can consume even 55x less memory than OnnxRuntime while being only 0.5-2x slower" The trade-off between (V)RAM use and inference time sounds like it could be advantageous in some scenarios, and not just when RAM is constrained like in the RPi case. I actually wonder if this weight unloading approach can be used to handle larger batch sizes in the same amount of RAM, in effect…

"0.5-2x slower" must be a typo on their part right? If something is 0.5x slower, then it is 2x faster. I assume they meant to say "1.5-2x slower".

hi,

I'm the author.

I have never questioned the clarity of that sentence, at least until today :-)

By "0.5x" I mean "0.5 times or 0.5 multiplied by the reference time" where "reference time" is the inference time of OnnxRuntime. So I'm actually meaning "50%".

I think the expression "0.5x slower", taken by itself, could be misleading, but in the context of the original sentence it becomes clearer ("while being only 0.5-2x slower", the "only" is important here!!!).

But I think the general context of that sentence defines its meaning. I am referring to the fact that under no circumstances could my project, with its premises, be in any scenario even a single millisecond faster than OnnxRuntime. Then in the second paragraph of the README the goal of the project is stated, which is precisely to trade off inference time for RAM usage! Obviously combined with the fact that the performance data is clearly reported and that I repeat several times that the generation of a single image takes hours or even dozens of hours.

However, given the possible misunderstanding, I will correct the sentence in the next few days.

Re: Running Stable Diffusion XL 1.0 in 298MB of RAM

#146

Post by gpt5 »

esskay wrote: post #124
madduci wrote: post #2

Amazing feat, but of course takes forever to generate an image (in the Readme states 11 hours)

It'd be interesting to see what the cost and power equivilence would be compared to a higher end method. I.e the time, cost (including all hardware required) and power taken to generate 100 images using 100 individual Pi Zero 2's (doesnt even need to be a W) vs something like an average mid-tier PC. I'd assume the pc would still likely win. Something like a Pi 4 or 5 may be a better benchmark than the Zero 2 as I get…

A GPU can produce an image in about 1 second.

Re: Running Stable Diffusion XL 1.0 in 298MB of RAM

#147

Post by esskay »

gpt5 wrote: post #146
esskay wrote: post #124

Earlier quoted context omitted.

It'd be interesting to see what the cost and power equivilence would be compared to a higher end method. I.e the time, cost (including all hardware required) and power taken to generate 100 images using 100 individual Pi Zero 2's (doesnt even need to be a W) vs something like an average mid-tier PC. I'd assume the pc would still likely win. Something like a Pi 4 or 5 may be a better benchmark than the Zero 2 as I get…

A GPU can produce an image in about 1 second.

That depends on the GPU, I'm not talking mid-high end (e.g RTX level), just your average 'basic' GPU.

Re: Running Stable Diffusion XL 1.0 in 298MB of RAM

#148

Post by robertlagrant »

vitoplantamura wrote: post #145
SigmundurM wrote: post #7

Earlier quoted context omitted.

"0.5-2x slower" must be a typo on their part right? If something is 0.5x slower, then it is 2x faster. I assume they meant to say "1.5-2x slower".

hi, I'm the author. I have never questioned the clarity of that sentence, at least until today :-) By "0.5x" I mean "0.5 times or 0.5 multiplied by the reference time" where "reference time" is the inference time of OnnxRuntime. So I'm actually meaning "50%". I think the expression "0.5x slower", taken by itself, could be misleading, but in the context of the original sentence it becomes clearer ("while being only 0.…

I still don't understand what you mean. 0.5 multiplied by the reference runtime is faster.

Do you mean it increases the runtime by 50%-200%?

Re: Running Stable Diffusion XL 1.0 in 298MB of RAM

#149

Post by anthk »

HappyDaoDude wrote: post #133
Archelaos wrote: post #117

Earlier quoted context omitted.

It reminds me of doing Mandelbrot fractals on my C64. Debugging my code was really hard.

I am still amazed by seeing Fractals rendered in real time. My Core 2 Duo can do the initial renders at about 1080p resolution in about a second or two. Something that would have taken hours to do on an Amiga in the 80's IF you even had the memory for that kind of storage.

I did 640x480 fractals at crazy speeds with Xaos on an AMD Athon, which was and is pretty well optimized.

Re: Running Stable Diffusion XL 1.0 in 298MB of RAM

#150

Post by vitoplantamura »

robertlagrant wrote: post #148
vitoplantamura wrote: post #145

Earlier quoted context omitted.

hi, I'm the author. I have never questioned the clarity of that sentence, at least until today :-) By "0.5x" I mean "0.5 times or 0.5 multiplied by the reference time" where "reference time" is the inference time of OnnxRuntime. So I'm actually meaning "50%". I think the expression "0.5x slower", taken by itself, could be misleading, but in the context of the original sentence it becomes clearer ("while being only 0.…

I still don't understand what you mean. 0.5 multiplied by the reference runtime is faster. Do you mean it increases the runtime by 50%-200%?

By "0.5x slower" I mean "50% slower" :-)
Post reply on HN