[flagged]
Are those 2048 x 2048 images still sensible? SD 1.5 is best used at 512x512 and may produce sensible images upto 768. It generates monstrosities above that. Similarly SD XL is good upto 1024.
Running Stable Diffusion XL 1.0 in 298MB of RAM
51–60 of 166 posts
Re: Running Stable Diffusion XL 1.0 in 298MB of RAM
#52Fascinating. The money quote: "OnnxStream can consume even 55x less memory than OnnxRuntime while being only 0.5-2x slower" The trade-off between (V)RAM use and inference time sounds like it could be advantageous in some scenarios, and not just when RAM is constrained like in the RPi case. I actually wonder if this weight unloading approach can be used to handle larger batch sizes in the same amount of RAM, in effect…
"0.5-2x slower" must be a typo on their part right? If something is 0.5x slower, then it is 2x faster. I assume they meant to say "1.5-2x slower".
It depends. What do you think if I say it's 1x slower?
Is it as fast as the original or does it take twice as long?
Re: Running Stable Diffusion XL 1.0 in 298MB of RAM
#53So this should be it for trying to regulate stable diffusion type tech, right? If these models and their inference infra can be shrunk down to be runnable on a PS2, it doesn't seem like it's possible to stop this tech without a totalitarian surveillance state (and barely even then!).
> If these models and their inference infra can be shrunk down to be runnable on a PS2, it doesn't seem like it's possible to stop this tech without a totalitarian surveillance state (and barely even then!).
The original requirement for these is 16GB of RAM, which can be had for less than $20. They run much faster on a GPU, which can be had for less than $200. Millions of ordinary people already have both of these things.
Re: Running Stable Diffusion XL 1.0 in 298MB of RAM
#54Re: Running Stable Diffusion XL 1.0 in 298MB of RAM
#55So this should be it for trying to regulate stable diffusion type tech, right? If these models and their inference infra can be shrunk down to be runnable on a PS2, it doesn't seem like it's possible to stop this tech without a totalitarian surveillance state (and barely even then!).
The war on general computing has been ongoing but not made enough inroads to stop people from owning general computing devices (yet)
Indeed, the death knell could be tolling not for regulation of ai but for general purposes computers. In AI we have four horsemen: copyright infringement, illegal pornography, fake news generation, and democratization of capabilities that large companies would rather monetize.
Re: Running Stable Diffusion XL 1.0 in 298MB of RAM
#56Earlier quoted context omitted.
I'm building exactly that with an eink display atm. Sadly, i can't seem to be able to build the XNNPACK stuff on my pi zero 2W in the repo...
that's an awesome idea, do you have a link to more information?
Ill write it up once it's done and post here, if it gains traction you might see it haha.
In all seriousness I can give a brief overview:
- I'll probably offload the image generation to the 5 year old intel nuc I already have as a home automation server, comfyUI in CPU mode takes 20-30 mins for a generation. Ideally it's all self contained on the Pi but that might be beyond me, skill wise.
- prompts are composed by taking time of day, season, special occasions (birthdays, xmas etc); adding random subjects from a long manually curated list; then asking gpt4 to creatively remix the prompt for variety
- i have an inky impression 7.3 inch 7 color eink display and a raspberry zero stuck onto it. Right now it'll simply download new images from the NUC every once in a while
- i like wood and i dislike the jagged 3d printer aesthetic so I'll create a frame from laser cut plywood by designing some stackable svg shapes in inkscape and sending those to a laser cutter
It works right now, functionally.
Considering that I'm painstakingly writing this on a phone with a sleeping 3 week old baby on my chest it'll be while before i have the energy to make it look like something you'd hang on your wall
Re: Running Stable Diffusion XL 1.0 in 298MB of RAM
#57So this should be it for trying to regulate stable diffusion type tech, right? If these models and their inference infra can be shrunk down to be runnable on a PS2, it doesn't seem like it's possible to stop this tech without a totalitarian surveillance state (and barely even then!).
Copyright infringement is quite cheap as well. Ease and illegality are tangential. You'd still stop commercial acts even if it's impossible to fully stop something.
That said, I don't think blanket regulation is all that likely anyhow.
Re: Running Stable Diffusion XL 1.0 in 298MB of RAM
#58Fascinating. The money quote: "OnnxStream can consume even 55x less memory than OnnxRuntime while being only 0.5-2x slower" The trade-off between (V)RAM use and inference time sounds like it could be advantageous in some scenarios, and not just when RAM is constrained like in the RPi case. I actually wonder if this weight unloading approach can be used to handle larger batch sizes in the same amount of RAM, in effect…
From my (albeit naive) reading, it doesn't appear that that they've reduced the amount of memory bandwidth required, simply the size of the working set required.
Since inference is generally memory bandwidth bound once you reach the level of 'does this model even fit in the given system', I'd imagine that this technique wouldn't help much for greater throughpit via larger batch sizes. Just one instance is probably already saturating the memory controller.
Maybe it'd help on the training side though?
Re: Running Stable Diffusion XL 1.0 in 298MB of RAM
#59[flagged]
Maybe next time shamelessly mention that you sell models for $29 and there is no instructions to convert from vanilla SD.
I can't believe this is still the top comment. I wish I didn't edit down my reply, shoulda just said "this is stupid, you're comparing your desktop to a raspberry pi"
ONNX streaming is way cooler and more impressive than another commercial wrapper around SD. Doesn't deserve this.
Re: Running Stable Diffusion XL 1.0 in 298MB of RAM
#60Earlier quoted context omitted.
"0.5-2x slower" must be a typo on their part right? If something is 0.5x slower, then it is 2x faster. I assume they meant to say "1.5-2x slower".
It depends. What do you think if I say it's 1x slower? Is it as fast as the original or does it take twice as long?
Twice as long. As fast as the original would be "0x slower" or "1x as fast".
People should just use duration instead of speed as you did at the end: "takes twice as long", "takes 1/3 of the time..."