Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Running Stable Diffusion XL 1.0 in 298MB of RAM

github.com

Post reply on HN
Print view View original

21–30 of 166 posts

Re: Running Stable Diffusion XL 1.0 in 298MB of RAM

#21

Post by lawlessone »

amelius wrote: post #13
lawlessone wrote: post #4

This would be really cool to have running embedded in a digital photo frame or wall painting.

Not very environment-friendly, though.

Compared to what though?

I think it might be friendlier in some aspects than fetching an image from a server running the big the models.

And you don't have to worry about service disruptions or api keys

An e-ink display doing it should only use energy when refreshes. And you could minimize refreshes to once a day, week, etc

Less friendly than a photo of course.

Re: Running Stable Diffusion XL 1.0 in 298MB of RAM

#23

Post by refulgentis »

Filligree wrote: post #22
symisc_devel wrote: post #20

[flagged]

Okay... what's the downside?

In terms of, what's the tradeoff for the time decrease?

Apples to oranges, they're comparing 11 hours on a Raspberry Pi Zero to:

- 10 seconds on Intel i7-13700

- 3 seconds on Intel i9-9990XE

- 5 seconds on Ryzen 9-5900X

Additionally, the 2048 is accomplished by using RealESRGAN to 2x, which isn't close to what a native 2048 diffuser's quality would be.

It does look interesting and is an achievement, in terms of, it's hard to write this stuff from scratch, much less in pure C++ without relying on GPU.

Re: Running Stable Diffusion XL 1.0 in 298MB of RAM

#24

Post by tavavex »

phh wrote: post #19
samr71 wrote: post #9

So this should be it for trying to regulate stable diffusion type tech, right? If these models and their inference infra can be shrunk down to be runnable on a PS2, it doesn't seem like it's possible to stop this tech without a totalitarian surveillance state (and barely even then!).

So this should be it for trying to regulate theft, right? If you can open a window without any tool other than your own body. It doesn't seem like it's possible to stop thefts without a totalitarian surveillance state (and barely event then!). Or same can be said about media "piracy". Or ransomwares. States have forever regulated things that are not possible to enforce purely technically.

But theft is quite a different thing, is it not? It's a physical act that someone can be caught engaging in - be it by another person, a guard or a security camera. Sure, the "barrier for entry" to commit it is low, but retailers et al. are doing as much as they can to raise it.

Piracy most often isn't treated as a criminal matter, but a civil one - few countries punish piracy severely, but companies are allowed to sue the pirate.

I agree with OP in principle - regulating generative AI use would be way harder than piracy or whatever, especially since all of it can be done purely locally and millions of people already have the software downloaded. And that's not getting into the reasoning behind a ban - piracy and similar "digital crimes" are banned because they directly harm someone, while someone launching Stable Diffusion on their PC doesn't do much of anything.

Re: Running Stable Diffusion XL 1.0 in 298MB of RAM

#26

Post by aftbit »

amelius wrote: post #13
lawlessone wrote: post #4

This would be really cool to have running embedded in a digital photo frame or wall painting.

Not very environment-friendly, though.

Why do you say that? The energy usage of inference? I would guess that the embodied energy of the digital photo frame is probably higher.

Re: Running Stable Diffusion XL 1.0 in 298MB of RAM

#27

Post by ShrigmaMale »

phh wrote: post #19
samr71 wrote: post #9

So this should be it for trying to regulate stable diffusion type tech, right? If these models and their inference infra can be shrunk down to be runnable on a PS2, it doesn't seem like it's possible to stop this tech without a totalitarian surveillance state (and barely even then!).

So this should be it for trying to regulate theft, right? If you can open a window without any tool other than your own body. It doesn't seem like it's possible to stop thefts without a totalitarian surveillance state (and barely event then!). Or same can be said about media "piracy". Or ransomwares. States have forever regulated things that are not possible to enforce purely technically.

no, this is a lousy analogy because there is a clear harm to others in the case of theft. we've tried regulating other difficult to regulate things where the harm is unclear or indirect (drugs being a good example) to no avail.

your piracy example is better. consider that it's the rise of more convenient options (netflix and spotify) not some effective policy that curtailed the prevalence of piracy.

Re: Running Stable Diffusion XL 1.0 in 298MB of RAM

#29

Post by Mockapapella »

omneity wrote: post #3

Fascinating. The money quote: "OnnxStream can consume even 55x less memory than OnnxRuntime while being only 0.5-2x slower" The trade-off between (V)RAM use and inference time sounds like it could be advantageous in some scenarios, and not just when RAM is constrained like in the RPi case. I actually wonder if this weight unloading approach can be used to handle larger batch sizes in the same amount of RAM, in effect…

I want this for LLMs. Having that much less of a memory footprint would allow us to put more models on a GPU at a time, and assuming the clock could keep up it could more than make up for the loss in inference speed per individual model

Re: Running Stable Diffusion XL 1.0 in 298MB of RAM

#30

Post by isoprophlex »

lawlessone wrote: post #4

This would be really cool to have running embedded in a digital photo frame or wall painting.

I'm building exactly that with an eink display atm. Sadly, i can't seem to be able to build the XNNPACK stuff on my pi zero 2W in the repo...
Post reply on HN