Running Stable Diffusion XL 1.0 in 298MB of RAM
1–10 of 166 posts
Re: Running Stable Diffusion XL 1.0 in 298MB of RAM
#2Amazing feat, but of course takes forever to generate an image (in the Readme states 11 hours)
Re: Running Stable Diffusion XL 1.0 in 298MB of RAM
#3Fascinating. The money quote:
"OnnxStream can consume even 55x less memory than OnnxRuntime while being only 0.5-2x slower"
The trade-off between (V)RAM use and inference time sounds like it could be advantageous in some scenarios, and not just when RAM is constrained like in the RPi case.
I actually wonder if this weight unloading approach can be used to handle larger batch sizes in the same amount of RAM, in effect increasing throughput massively at the cost of latency.
Re: Running Stable Diffusion XL 1.0 in 298MB of RAM
#4This would be really cool to have running embedded in a digital photo frame or wall painting.
Re: Running Stable Diffusion XL 1.0 in 298MB of RAM
#5[dead]
Re: Running Stable Diffusion XL 1.0 in 298MB of RAM
#6I can't wait for Stable Diffusion for Windows 3.1
Re: Running Stable Diffusion XL 1.0 in 298MB of RAM
#7Fascinating. The money quote: "OnnxStream can consume even 55x less memory than OnnxRuntime while being only 0.5-2x slower" The trade-off between (V)RAM use and inference time sounds like it could be advantageous in some scenarios, and not just when RAM is constrained like in the RPi case. I actually wonder if this weight unloading approach can be used to handle larger batch sizes in the same amount of RAM, in effect…
"0.5-2x slower" must be a typo on their part right? If something is 0.5x slower, then it is 2x faster.
I assume they meant to say "1.5-2x slower".
Re: Running Stable Diffusion XL 1.0 in 298MB of RAM
#8This would be really cool to have running embedded in a digital photo frame or wall painting.
Great idea, where every 10 hours or so it would refresh with a new image it created itself (perhaps based on a theme supplied by the user).
Re: Running Stable Diffusion XL 1.0 in 298MB of RAM
#9So this should be it for trying to regulate stable diffusion type tech, right? If these models and their inference infra can be shrunk down to be runnable on a PS2, it doesn't seem like it's possible to stop this tech without a totalitarian surveillance state (and barely even then!).
Re: Running Stable Diffusion XL 1.0 in 298MB of RAM
#1011 hours remind me of doing raytracing on my Amiga 500 back in the day. It was definitely an overnight job for the "final" render.