Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Running Stable Diffusion XL 1.0 in 298MB of RAM

github.com

Post reply on HN
Print view View original

121–130 of 166 posts

Re: Running Stable Diffusion XL 1.0 in 298MB of RAM

#121

Post by pphysch »

Drakim wrote: post #85
pphysch wrote: post #65

Earlier quoted context omitted.

That is virtually impossible because Turing-complete systems are everywhere

Just like how making weed illegal is virtually impossible because anybody can grow marijuana in their backyard. How many regular people would risk owning turning-complete devices that can run unauthorized software if it would net you jail time if caught? Lots of countries are already itching towards banning VPN, corpo needs be damned. Especially now that the iPhone has shown having a device that can only run approved…

I'm more referring to the fact that stuff like PowerPoint and Minecraft and who knows what are Turing-complete, albeit with awful performance.

Theoretically, you can have a totally owned device managed by Big Brother, yet generate AI smut with a general purpose CPU built in PowerPoint.

How do you possibly regulate that?

Re: Running Stable Diffusion XL 1.0 in 298MB of RAM

#122

Post by yowlingcat »

pphysch wrote: post #65
Drakim wrote: post #47

Earlier quoted context omitted.

The war on general computing has been ongoing but not made enough inroads to stop people from owning general computing devices (yet)

That is virtually impossible because Turing-complete systems are everywhere

I wonder if there's an analogy to be made here to DRM. In theory, yes, DRM shouldn't be possible, but in practice, manufacturers have been able to hobble hardware acceleration behind trusted computing model. Often, they do a poor job and it gets cracked (as with HDCP [1], and UWP [2]).

The question in my head is whether the failures in their approaches are due to a flaw in the implementation (in which case it's practically possible to do what they're trying to do although they haven't figured out a way to do it), or whether it's fundamentally impossible. With DRM and content, there's always the analog hole, and if you have physical control over the device, there's always a way to crack the software and the hardware if need be. My questions are whether:

a) this is a workable analogy (I think it's imperfect because Gen AI and DRM are kinda different beasts)

b) even if it was, is there real way to limit Gen AI at a hardware level (I think that's also hard because as long as you can do hardware accelerated matmul it's basically opening up the equivalent of the analog hole towards semi-turing completeness which is also hardware accelerated)

I imagine someone has thought through this more deeply than me and would be curious what they think.

[1] https://en.wikipedia.org/wiki/High-bandwidth_Digital_Content...

[2] https://techaeris.com/2018/02/18/microsoft-uwp-protection-cr...

Re: Running Stable Diffusion XL 1.0 in 298MB of RAM

#124

Post by esskay »

madduci wrote: post #2

Amazing feat, but of course takes forever to generate an image (in the Readme states 11 hours)

It'd be interesting to see what the cost and power equivilence would be compared to a higher end method. I.e the time, cost (including all hardware required) and power taken to generate 100 images using 100 individual Pi Zero 2's (doesnt even need to be a W) vs something like an average mid-tier PC.

I'd assume the pc would still likely win.

Something like a Pi 4 or 5 may be a better benchmark than the Zero 2 as I get the impression its been used more for the challenge than practicality.

Re: Running Stable Diffusion XL 1.0 in 298MB of RAM

#126

Post by loondri »

The trade-off between memory usage and inference time uncovers a potential flaw in prioritizing resource efficiency over performance.

This would deter real-time or near real-time applications where latency is a critical factor.

Also, the confusion over the phrase "0.5-2x slower" highlights a possible lack of clarity in communication within the community, which would hinder the accurate assessment and adoption of such optimizations in practice.

Re: Running Stable Diffusion XL 1.0 in 298MB of RAM

#127

Post by refulgentis »

leonidasv wrote: post #120
refulgentis wrote: post #23

Earlier quoted context omitted.

In terms of, what's the tradeoff for the time decrease? Apples to oranges, they're comparing 11 hours on a Raspberry Pi Zero to: - 10 seconds on Intel i7-13700 - 3 seconds on Intel i9-9990XE - 5 seconds on Ryzen 9-5900X Additionally, the 2048 is accomplished by using RealESRGAN to 2x, which isn't close to what a native 2048 diffuser's quality would be. It does look interesting and is an achievement, in terms of, it's…

Unrelated, but now I'm curious about how much would it take on RPis 4 and 5.

yeah me too...I've been very negative about the edge, it got overhyped with the romanticization of local LLMs, but there's a bunch of stuff coming together at the same time...Raspberry Pi 5...Mistral 7B Orca is my 20th try of a local LLM...and the first time it handled simple conversation with RAG. And new diffusion, even every 2 hours, is a credible product, arguing about power consumption aside...

Re: Running Stable Diffusion XL 1.0 in 298MB of RAM

#128

Post by josephg »

loondri wrote: post #126

The trade-off between memory usage and inference time uncovers a potential flaw in prioritizing resource efficiency over performance. This would deter real-time or near real-time applications where latency is a critical factor. Also, the confusion over the phrase "0.5-2x slower" highlights a possible lack of clarity in communication within the community, which would hinder the accurate assessment and adoption of such…

You might be making some good points, but it took me about 3 attempts to understand your comment.

For example:

> Also, the confusion over the phrase "0.5-2x slower" highlights a possible lack of clarity in communication within the community, which would hinder the accurate assessment and adoption of such optimizations in practice.

Maybe instead:

> The phrase "0.5-2x slower" is confusing. You might get more adoption if the language was more clear.

Re: Running Stable Diffusion XL 1.0 in 298MB of RAM

#129

Post by smusamashah »

dragonwriter wrote: post #86
smusamashah wrote: post #51

Earlier quoted context omitted.

Are those 2048 x 2048 images still sensible? SD 1.5 is best used at 512x512 and may produce sensible images upto 768. It generates monstrosities above that. Similarly SD XL is good upto 1024.

> Are those 2048 x 2048 images still sensible? SD 1.5 is best used at 512x512 and may produce sensible images upto 768. It generates monstrosities above that. Similarly SD XL is good upto 1024. You can do significantly higher resolutions with various tricks like tiled diffusion, which is also a memory efficiency hack. (The stable-diffusion-webui tiled diffusion extension uses 2560×1280 direct [no upscale step] genera…

Up scaling the image in chunk creates loads of semantic issues. For example, bottom of tree might look further in the mountains but it's top will be near you. You don't see problems like these in non scaled images.

Re: Running Stable Diffusion XL 1.0 in 298MB of RAM

#130

Post by mr_toad »

codetrotter wrote: post #83
pmarreck wrote: post #66

Earlier quoted context omitted.

This is more than a little melodramatic. https://frame.work/ and the https://mntre.com/ MNT Reform: Exist

If my country decides to ban the ownership of general purpose computers for individual persons, they would order the customs service to stop import of any computer hardware that enabled general purpose computing. Now I would not be able to have any computer shipped to me from outside my country, so I could no longer buy from either of those vendors you linked. Furthermore, it also would mean that I would not be able…

People would take the CPUs out of other devices and use them. A consumer grade router has most of the hardware you need to make a general purpose computer.
Post reply on HN