Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

OnnxStream: Stable Diffusion XL 1.0 Base on a Raspberry Pi Zero 2

github.com

Post reply on HN
Print view View original

31–40 of 88 posts

Re: OnnxStream: Stable Diffusion XL 1.0 Base on a Raspberry Pi Zero 2

#31

Post by godelski »

Zetobal wrote: post #21

I know 29 minutes is long but theoretically you can have all the images you ever want in a small 6gb package and run inference on (nearly) everything. That's fucking amazing.

But honest question, if this is your goal, why not use a GAN instead? You should still be able to produce high quality images but at a much faster rate (I'd guess around 10 minutes?). Sure, you'll have a bit lower diversity and maybe not SOTA quality image generation, but neither is this thing. Or you could reduce quality. This reddit user seems to be doing fast inference on a pi[0] using stylegan, but that's before mobile stylegan came out which uses Just seems like different models, different contexts. Certainly you'd want diffusion on the computer you're doing photoshop on, but random images? Different context.

[0] https://www.reddit.com/r/raspberry_pi/comments/hf7lbh/i_made...

Re: OnnxStream: Stable Diffusion XL 1.0 Base on a Raspberry Pi Zero 2

#32

Post by Lerc »

Nice to see people finding way to get the square peg though the round hole.

Something that I wondered when the Raspberry Pi 5 came out is the weirdness that might be possible now that they have their own chip doing IO cleverness.

On the PI 5, the two MIPI interfaces ave been enhanced to do either output or input, It made me wonder if the ports are now generalized enough that you could daisy chain a string of PI 5's connecting MIPI to MIPI. Then you could run inference layers on individual PIs and pass the activations down the MIPI. 10x 8Gig Pi 5's might not be the speediest way to get an 80gig setup, but it would certainly be the cheapest (for now)

Re: OnnxStream: Stable Diffusion XL 1.0 Base on a Raspberry Pi Zero 2

#33

Post by omgwtfbyobbq »

filterfiber wrote: post #14
m3kw9 wrote: post #3

Could be a nice wall paper generator every 29 min. Input a big list of random prompts and let it rotate

This project is a fun POC but it's not very practical for that type of application. A 4090 can generate over 100 images a second with turbo+lcm and a few techniques, you can make 2 days worth of images in 1 seconds. You could make a years worth in roughly 3 minutes and put them on the sd card

Do your have references for that?

I found this claiming an A100 can generate 1 image/s.

https://oneflow2020.medium.com/text-to-image-in-less-than-1-...

Re: OnnxStream: Stable Diffusion XL 1.0 Base on a Raspberry Pi Zero 2

#34

Post by omgwtfbyobbq »

filterfiber wrote: post #18
Sharlin wrote: post #15

Earlier quoted context omitted.

But that's not the point, obviously. Sometimes, being slow is a feature. Besides, a 4090 costs more than a small car.

> But that's not the point, obviously. If you want to say the zero2-w is what's making it then sure. > Besides, a 4090 costs more than a car. They only cost ~0.70USD for 1 hr. In fact you could put this on an A100 for 1$/hr. Renting would make the most sense for this type of thing.

It depends on what you're using the images for.

If there's a human in the loop, 100 images/s is likey too much volume, especially if prompt engineering is needed.

At the same time, 2 images/hr is way too slow.

Re: OnnxStream: Stable Diffusion XL 1.0 Base on a Raspberry Pi Zero 2

#35

Post by dang »

practice9 wrote: post #30
dang wrote: post #27

Submitted title was "Stable Diffusion Turbo on a Raspberry Pi Zero 2 generates an image in 29 minutes", which is good to know in order to understand some of the comments posted before I changed the title. Submitters: if you want to say what you think is important about an article, that's fine, but do it by adding a comment to the thread. Then your view will be on a level playing field with everyone else's: https://hn…

Wait, but the new title doesn't seem to be correct

We certainly want it to be correct! I took "Stable Diffusion XL 1.0 Base on a Raspberry Pi Zero 2" from the About part of https://github.com/vitoplantamura/OnnxStream/tree/c0cb4b3d7b.... Is it wrong?

Re: OnnxStream: Stable Diffusion XL 1.0 Base on a Raspberry Pi Zero 2

#36

Post by godelski »

taf2 wrote: post #7

So it’s safe to assume in the next 10 years - AI will be running locally on every device from phones, laptops and even many embedded devices. Even robots- from street cleaning bots to helpful human assistants?

Yes and no. Context matters.

Will models of similar quality to the current LLaMA, GPT, and Stable Diffusion be running locally on devices and edge systems? Very likely.

Will much higher quality models that still require compute incapable by such edge or consumer devices be available, sold as a service, and in high use? Also very likely.

So expect current quality to make it to your devices but don't expect to necessarily move everything to local because the whole ecosystem will improve too. Overton window will shift and it's like asking if gaming will move to phones. In some ways yes, in other ways you're still going to want to buy that Playstation/Xbox/PC.

Re: OnnxStream: Stable Diffusion XL 1.0 Base on a Raspberry Pi Zero 2

#38

Post by johnklos »

filterfiber wrote: post #14
m3kw9 wrote: post #3

Could be a nice wall paper generator every 29 min. Input a big list of random prompts and let it rotate

This project is a fun POC but it's not very practical for that type of application. A 4090 can generate over 100 images a second with turbo+lcm and a few techniques, you can make 2 days worth of images in 1 seconds. You could make a years worth in roughly 3 minutes and put them on the sd card

It's so nice of you to offer to buy 4090 cards for people who can only otherwise afford Raspberry Pis ;)

Re: OnnxStream: Stable Diffusion XL 1.0 Base on a Raspberry Pi Zero 2

#40

Post by GaggiX »

godelski wrote: post #31
Zetobal wrote: post #21

I know 29 minutes is long but theoretically you can have all the images you ever want in a small 6gb package and run inference on (nearly) everything. That's fucking amazing.

But honest question, if this is your goal, why not use a GAN instead? You should still be able to produce high quality images but at a much faster rate (I'd guess around 10 minutes?). Sure, you'll have a bit lower diversity and maybe not SOTA quality image generation, but neither is this thing. Or you could reduce quality. This reddit user seems to be doing fast inference on a pi[0] using stylegan, but that's before…

The quality is not really close, also StyleGAN2 is not conditioned on text.
Post reply on HN