Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Running Stable Diffusion XL 1.0 in 298MB of RAM

github.com

Post reply on HN
Print view View original

11–20 of 166 posts

Re: Running Stable Diffusion XL 1.0 in 298MB of RAM

#12

Post by [deleted] »

SigmundurM wrote: post #7
omneity wrote: post #3

Fascinating. The money quote: "OnnxStream can consume even 55x less memory than OnnxRuntime while being only 0.5-2x slower" The trade-off between (V)RAM use and inference time sounds like it could be advantageous in some scenarios, and not just when RAM is constrained like in the RPi case. I actually wonder if this weight unloading approach can be used to handle larger batch sizes in the same amount of RAM, in effect…

"0.5-2x slower" must be a typo on their part right? If something is 0.5x slower, then it is 2x faster. I assume they meant to say "1.5-2x slower".

[deleted]

Re: Running Stable Diffusion XL 1.0 in 298MB of RAM

#15

Post by dahart »

SigmundurM wrote: post #7
omneity wrote: post #3

Fascinating. The money quote: "OnnxStream can consume even 55x less memory than OnnxRuntime while being only 0.5-2x slower" The trade-off between (V)RAM use and inference time sounds like it could be advantageous in some scenarios, and not just when RAM is constrained like in the RPi case. I actually wonder if this weight unloading approach can be used to handle larger batch sizes in the same amount of RAM, in effect…

"0.5-2x slower" must be a typo on their part right? If something is 0.5x slower, then it is 2x faster. I assume they meant to say "1.5-2x slower".

Maybe they meant 50%-200% slower, in which case the x-factor range would really be 1.5x to 3x?

Re: Running Stable Diffusion XL 1.0 in 298MB of RAM

#18

Post by agilob »

amelius wrote: post #13
lawlessone wrote: post #4

This would be really cool to have running embedded in a digital photo frame or wall painting.

Not very environment-friendly, though.

Dude, we are literally using single use plastic bottles to store water for a few weeks in them.

Re: Running Stable Diffusion XL 1.0 in 298MB of RAM

#19

Post by phh »

samr71 wrote: post #9

So this should be it for trying to regulate stable diffusion type tech, right? If these models and their inference infra can be shrunk down to be runnable on a PS2, it doesn't seem like it's possible to stop this tech without a totalitarian surveillance state (and barely even then!).

So this should be it for trying to regulate theft, right? If you can open a window without any tool other than your own body. It doesn't seem like it's possible to stop thefts without a totalitarian surveillance state (and barely event then!).

Or same can be said about media "piracy". Or ransomwares.

States have forever regulated things that are not possible to enforce purely technically.

Post reply on HN