This is quite impressive, thank you for sharing it!
OnnxStream: Stable Diffusion XL 1.0 Base on a Raspberry Pi Zero 2
61–70 of 88 posts
Re: OnnxStream: Stable Diffusion XL 1.0 Base on a Raspberry Pi Zero 2
#62Earlier quoted context omitted.
I was just using that as a reference. Stable diffusion will run well with almost any relatively modern gpu. You don't have to use a 4090, you'll still get double digit performance with a 3060 or whatnot. > for people who can only otherwise afford Raspberry Pis ;) You can rent a 4090 for 0.7USD/1hr, or get an A100 for 1.1USD/hr. And if your project is a display + raspberry pi then those costs will dwarf the rental cos…
You can use Google Colab for free as well.
I tried to load up atomic yesterday to see how it has changed since early in the year.
It failed to load. They reported that it was detected as against their terms of service.
Re: OnnxStream: Stable Diffusion XL 1.0 Base on a Raspberry Pi Zero 2
#63Earlier quoted context omitted.
> I didn't say any of that I'm sorry, then I don't know what you're saying. Because what I read was 2 claims. 1) GAN quality is less than diffusion. 2) GANs can't do T2I. I think I adequately showed that both these assumptions were wrong. I'm not sure what else your comment meant as that contains all of its words... > there is simply no open source GAN model that can compete with the open source diffusion models we h…
Just FYI, it will less obvious that you’re entrenched and “dug in” to the field of GAN research if you’re less defensive when people say fairly reasonable things about different ideas.
Problem is none of what you believe about me is true. By main focus area is explicit density estimation, GANs are implicit. I like math...
But you can check my comment history to see that I really hate hype. I said there are better models for different tasks. Here's a breakdown:
- Autoregressive: Best for time sequential data
- Normalizing Flows: Best for density estimation and statistics
- Diffusion: Best at general image synthesis, editing, and diversity.
- GANs: Best when working on edge devices or throughput is critical
- VAEs: Best for situations between diffusion and flows, where a implicit PCA is desired
You can tell why I discussed GANs because it hit the specific use case. Remember my first comment is literally saying the motivation is trading image quality and diversity for generation time. It's because again, the claim is that there isn't one size fits all models. Such a notion is silly. You don't use a diffusion model to do real time image upscaling (e.g. for video) nor do you want it for RTX supersampling, but you do want diffusion for general image synthesis tasks in such areas like photoshop. Inpainting, outpainting, T2I, unconditional, I2I, and all that you want diffusion because it is much better for those tasks.
Just... diffusion requires big models and they're slow and the raspberry pi is small. So you make sacrifices.
Re: OnnxStream: Stable Diffusion XL 1.0 Base on a Raspberry Pi Zero 2
#64Earlier quoted context omitted.
> I didn't say any of that I'm sorry, then I don't know what you're saying. Because what I read was 2 claims. 1) GAN quality is less than diffusion. 2) GANs can't do T2I. I think I adequately showed that both these assumptions were wrong. I'm not sure what else your comment meant as that contains all of its words... > there is simply no open source GAN model that can compete with the open source diffusion models we h…
Sorry, but at the link you provided there is no model weights, only inferred images offered for examination, so at this point it is at vaporware stage compared to stable diffusion tools.
Sorry, you're right. That is a really weird thing to do. Just provide 6GB of images...? They even don't have an issues page. Looks to be a common thing by that author. You're right, very suspicious.
Lafite has their checkpoints at least. Results aren't great, but it is small and fast.
They have a colab but it's broken (lol). Fix by removing the torch versions, and add gdown. Replace the wget line with "!gdown https://drive.google.com/u/0/uc?id=17ER7Yl02Y6yCPbyWxK_tGrJ8..." (checkpoint from from their github). Then everything will work fine. It took some time to get some decent outputs (but then again so did my first time with any diffusion model. This is definitely lower quality though). At least the authors look engaged in the github issues and do show how to get better results. (Always be suspicious of the images shown in papers... this one is certainly no exception)
Re: OnnxStream: Stable Diffusion XL 1.0 Base on a Raspberry Pi Zero 2
#65Earlier quoted context omitted.
> I didn't say any of that I'm sorry, then I don't know what you're saying. Because what I read was 2 claims. 1) GAN quality is less than diffusion. 2) GANs can't do T2I. I think I adequately showed that both these assumptions were wrong. I'm not sure what else your comment meant as that contains all of its words... > there is simply no open source GAN model that can compete with the open source diffusion models we h…
>I'm sorry, then I don't know what you're saying I mean it was pretty simple, the quality is not really close, also StyleGAN2 is not conditioned on text (because you talked about this model in first comment). In the future there could be one that is competitive but not today. GigaGAN the best GAN model by far is not open source and it's not competitive yet because even the images cherry picked for the paper and proje…
Look, every single message I've said that you're trading quality for speed. That does in fact mean GANs are worse. I'm not sure why you think I have said anything short of that, I've explicitly agreed with you that diffusion produces higher quality images, and I'm not interested in repeating myself any further.
We're talking about a fucking raspberry pi, it is definitely reasonable to want to __trade quality for speed__ when you're working on a tiny computer.
I'm glad you're aware that FID has limits. For some reason this is uncommon. But there are more limits than the backbone classifier. Yes, DINOv2 helps (so does CLIP-FID and clean FID (which is backbone independent. I feel I need to explain because we're having difficulties communicating and going to their code and seeing they use inception isn't going to mean anything to be because that's not what that work is about)), but fundamentally the difference between two normalization layers of a classifier is not actually measurement of image quality. It correlates, but these also are fundamentally about the distributional nature of the outputs. There actually is no method that does particularly well but we're over here doing our best. Just got to be aware of the limits of your metrics because (as is the theme of our entire conversation), context matters. Here's a paper you may find interesting https://arxiv.org/abs/2306.04675
Re: OnnxStream: Stable Diffusion XL 1.0 Base on a Raspberry Pi Zero 2
#66Earlier quoted context omitted.
This project is a fun POC but it's not very practical for that type of application. A 4090 can generate over 100 images a second with turbo+lcm and a few techniques, you can make 2 days worth of images in 1 seconds. You could make a years worth in roughly 3 minutes and put them on the sd card
Do your have references for that? I found this claiming an A100 can generate 1 image/s. https://oneflow2020.medium.com/text-to-image-in-less-than-1-...
They're might be talking about this[0] as it has been popular recently. It can definitely do >60 imgs/s on my 3080Ti, but you're not going to want any of those images. They are absolute garbage.[1] I can do a little under an image a second and some may be quite usable, but nowhere near what you're going to get from the standard model.
[0] https://github.com/aifartist/ArtSpew/
[1] but the project is still cool, just context...
Re: OnnxStream: Stable Diffusion XL 1.0 Base on a Raspberry Pi Zero 2
#67Earlier quoted context omitted.
Do your have references for that? I found this claiming an A100 can generate 1 image/s. https://oneflow2020.medium.com/text-to-image-in-less-than-1-...
> I found this claiming an A100 can generate 1 image/s. The article you linked is over a year old. Needless to say there have been a LOT of optimizations in the last year. Back then it was common to use 50+ steps for many of the common samplers. Current methods use a few steps like 1. This OnnxStream are using SDXL-turbo, and you can combine LCM and a few other methods to go very fast. The reason it's so much faster…
> This repo claims 149 images/s on a 4090
I actually get around 100 imgs/s on my 3080Ti. Three things to note: 1) you gotta run the max perf code to get the high throughput, 2) the images in this setting are absolute garbage, 3) you don't save the images so you're going to have to edit the code to extract them.
Definitely agree that this project is much more about the cool factor. I suggested a GAN in other comment for similar reasoning (because it's a pi...) but if you want quality images well I'm not sure why anyone would expect to get those out of a pi. High quality images take time and skill. But it's also HN, I'm all for doing things for the cool factor (as long as we don't sell them as things they aren't. ML is cool enough that it doesn't need all the hype and garbage)
Re: OnnxStream: Stable Diffusion XL 1.0 Base on a Raspberry Pi Zero 2
#68Earlier quoted context omitted.
Do your have references for that? I found this claiming an A100 can generate 1 image/s. https://oneflow2020.medium.com/text-to-image-in-less-than-1-...
> I found this claiming an A100 can generate 1 image/s. The article you linked is over a year old. Needless to say there have been a LOT of optimizations in the last year. Back then it was common to use 50+ steps for many of the common samplers. Current methods use a few steps like 1. This OnnxStream are using SDXL-turbo, and you can combine LCM and a few other methods to go very fast. The reason it's so much faster…
> Back then it was common to use 50+ steps for many of the common samplers. Current methods use a few steps like 1.
The "look how fast we can go" method (turbo model with 1 step and without CFG) is blindingly fast, but the quality is...nothing close to what was being done in normal 50+ steps with normal setitngs gens.
Realistically, even with Turbo+LCM, you're still going to 4+ steps (often 8+), with CFG, for reasonable one-generation quality anywhere close to the images people generated at 50+ steps without Turbo/LCM.
Which is still a big improvement in speed.
Re: OnnxStream: Stable Diffusion XL 1.0 Base on a Raspberry Pi Zero 2
#69Earlier quoted context omitted.
>I'm sorry, then I don't know what you're saying I mean it was pretty simple, the quality is not really close, also StyleGAN2 is not conditioned on text (because you talked about this model in first comment). In the future there could be one that is competitive but not today. GigaGAN the best GAN model by far is not open source and it's not competitive yet because even the images cherry picked for the paper and proje…
Look, every single message I've said that you're trading quality for speed. That does in fact mean GANs are worse. I'm not sure why you think I have said anything short of that, I've explicitly agreed with you that diffusion produces higher quality images, and I'm not interested in repeating myself any further. We're talking about a fucking raspberry pi, it is definitely reasonable to want to __trade quality for spee…
Running SD v1.5 using one single step is faster than sampling with GigaGAN and you can achieve better coherency in my opinion: https://arxiv.org/abs/2311.09257
Then it does make more sense to run this on a raspberry pi.
Re: OnnxStream: Stable Diffusion XL 1.0 Base on a Raspberry Pi Zero 2
#70Earlier quoted context omitted.
Look, every single message I've said that you're trading quality for speed. That does in fact mean GANs are worse. I'm not sure why you think I have said anything short of that, I've explicitly agreed with you that diffusion produces higher quality images, and I'm not interested in repeating myself any further. We're talking about a fucking raspberry pi, it is definitely reasonable to want to __trade quality for spee…
Running SD v1.5 using one single step is faster than sampling with GigaGAN and you can achieve better coherency in my opinion: https://arxiv.org/abs/2311.09257 Then it does make more sense to run this on a raspberry pi.
Yeah I'll agree to that. But neither UFOGen nor GigaGAN have released models (I've admitted to being wrong about GigaGAN's checkpoints. In fairness, who the fuck releases 6GBs of generated images? And removes the issues tab from GitHub? Can we agree that's sketchy as fuck?[0]).
LAFITE is the only of the 3 with released checkpoints but it's definitely not to the same quality (Nvidia backed down to place it in products and imo a bad move). But it is 75M params compared to the 1B of UFOGen and GGAN. I was able to get a bit better images by some prompt engineering but yeah, classic paper painting a much better picture of their model than it actually is (pun intended). But then again, I got shitty images the first time I used SD so YMMV. There's probably better works out there but honestly my focus is elsewhere and no one's got time to keep on top of and try everything coming out.
I'm literally just saying that you can trade quality for inference. Do you disagree?
Because I'm not sure why you think I think GANs are better for quality, as I've said the opposite many times. Why are you hyper-focused on quality and thinking I've said GANs do better. This is why we're talking past one another, because you're attributing assumptions to me that I ('m doing my fucking best to clarify that I) don't have. If you think I think GANs produce higher quality images, I assure you that this is from your imagination as I've never stated such and it is not an opinion that I have. Sorry, I said I wouldn't say this again, so last time for real.
[0] There's too much sketchy shit going on in ML research right now and honestly, that's why I've been more passionate about trying to get people to think harder and about context. Again, this is about context. (I also really hate this railroading as it stifles community innovation and sweeps important problems under the rug by saying to just rely on large companies for checkpoints. The community doing so much around Stable Diffusion is awesome, but I want to see that around lots of works because there's a lot that can be accelerated by even a hundredth of this community effort)