Live data from Hacker News

Stable Diffusion with Core ML on Apple Silicon

machinelearning.apple.com

91–100 of 184 posts

Re: Stable Diffusion with Core ML on Apple Silicon

#91

This may sound naive, but what are some use cases of running SD models locally? If the free/cheap options exist (like running SD on powerful servers), then what's the advantage of this new method?

Don't want to take a risk to be banned by generating some images like nsfw

Re: Stable Diffusion with Core ML on Apple Silicon

#92
post #21
post #7

Earlier quoted context omitted.

i'm very ignorant here so forgive me but if it can generate images that fast can it be used to generate a video?

Video is really a series of frames, the framerate for film/human can get away with 24 frames/second-- so maybe ~40ms/image for real-time at least? What's cool about the era in which we live is if you look at high-performance graphics for games or simulations, for instance, it may in fact be faster to a the model to "enhance" a low-resolution frame rather than trying to render it fully on the machine. ex. AMD's FSR vs…

You can generate video a lot more efficiently than frame by frame. For example, you can generate every other frame and use something like DLSS 3.0 to fill in the missing ones.

Re: Stable Diffusion with Core ML on Apple Silicon

#93
post #14
post #2

Atila from Apple on the expected performance: > For distilled StableDiffusion 2 which requires 1 to 4 iterations instead of 50, the same M2 device should generate an image in https://twitter.com/atiorh/status/1598399408160342039

With the full 50 iterations it appears to be about 30s on M1. They have some benchmarks on the github repo: https://github.com/apple/ml-stable-diffusion For reference, previously I was getting about <3 minutes for 50 iterations on my Macbook Air M1. I haven't yet tried Apple's implementation but it looks like a huge improvement. It might take it from "possible" to "usable".

How do dreamstudio/craiyon/hugging face manage to do seemingly quicker on their interfaces? Are they hosting these models on super beefy and costly GPUs for free?

Re: Stable Diffusion with Core ML on Apple Silicon

#94
post #80

Earlier quoted context omitted.

What part of the SD license prohibits that?

No part of it.

I mean, it is a legal time bomb in general[0], with a non-standard license that has special stipulations in an amendment. Do you really incur the weeks of lead time that it would take Legal to review the legality of redistributing this model?

0: https://github.com/CompVis/stable-diffusion/blob/main/LICENS...

Re: Stable Diffusion with Core ML on Apple Silicon

#95
post #87
post #56

Earlier quoted context omitted.

I'm baffled by continued hate on Electron. The option isn't between Electron and a lean OS-native application, but between Electron and nothing. I can build an Electron app in under a day with a pretty UI. It would take me several months to get anything sensible that is OS native. And I'm not going to sit down and learn the alternative. So please just say "thank you" to the developers that are sharing free things wit…

While I agree the posters comment felt entitled, it should be possible to pick up and make a SwiftUI version of the app fairly quickly. I assume the developer went for electron due to familiarity, but it would be a pretty good exercise for someone to port it to SwiftUI and native Swift for the front end. I would do it myself but sadly am bound by other clauses.

It reminds me "Teach Yourself C++ in 21 days". You just need to quickly learn Swift (which you will use exactly nowhere after this task).

It's astonishing how ungrateful people are. Even writing documentation for the software is quite a time-consuming action - writing the software itself is much more time-consuming.

So you are looking at some free software, that gives you the ability to play with StableDiffusion in 2 clicks, has a wide range of features and settings, surely required a ton of time to implement, and you arrogantly saying “pff, an Electron app...”

Re: Stable Diffusion with Core ML on Apple Silicon

#96
post #63

There's also https://draw.nnc.ai/ - which is an iOS / iPad app running Stable Diffusion. The author has a detailed blogpost outlining how he modified the model to use Metal on iOS devices. https://liuliu.me/eyes/stretch-iphone-to-its-limit-a-2gib-mo...

Yeah, that's what immediately came to mind for me as well. I don't know how similar/different the two solutions are, but it made me smile a bit that what Apple is showing off here has been already done by a single independent developer :)

Re: Stable Diffusion with Core ML on Apple Silicon

#97
post #93
post #14

Earlier quoted context omitted.

With the full 50 iterations it appears to be about 30s on M1. They have some benchmarks on the github repo: https://github.com/apple/ml-stable-diffusion For reference, previously I was getting about <3 minutes for 50 iterations on my Macbook Air M1. I haven't yet tried Apple's implementation but it looks like a huge improvement. It might take it from "possible" to "usable".

How do dreamstudio/craiyon/hugging face manage to do seemingly quicker on their interfaces? Are they hosting these models on super beefy and costly GPUs for free?

M1's single-threaded CPU performance and power efficiency are exceptional; however M1's GPU performance is nothing special compared to normal discrete GPUs. You don't need something super beefy to beat M1 on the GPU side.

But also yes, it's gotta be expensive to host these models and I'm not sure where all these subsidies are coming from. I expect that we'll eventually see these things transition to more paid services.

Re: Stable Diffusion with Core ML on Apple Silicon

#98

How come you always have to install some version of pytorch or tensor flow to run these ml models? When I'm only doing inference shouldn't there be easier ways of doing that, with automatic hardware selection etc. Why aren't models distributed in a standard format like onnx, and inference on different platforms solved once per platform?

Seconded, I wish for a way to work with ML models using native code rather than through some Python scripting interface. I believe TensorFlow is there with C++, but it works only with C++ and not through FFI.

Re: Stable Diffusion with Core ML on Apple Silicon

#99
post #87

Earlier quoted context omitted.

While I agree the posters comment felt entitled, it should be possible to pick up and make a SwiftUI version of the app fairly quickly. I assume the developer went for electron due to familiarity, but it would be a pretty good exercise for someone to port it to SwiftUI and native Swift for the front end. I would do it myself but sadly am bound by other clauses.

It reminds me "Teach Yourself C++ in 21 days". You just need to quickly learn Swift (which you will use exactly nowhere after this task). It's astonishing how ungrateful people are. Even writing documentation for the software is quite a time-consuming action - writing the software itself is much more time-consuming. So you are looking at some free software, that gives you the ability to play with StableDiffusion in 2…

I think you completely misunderstood what I was saying.

I wasn’t saying that the author of DiffusionBee should make a SwiftUI application. In fact I said the opposite in that I agree that the person who expected a native app is entitled.

I was however refuting the person I was responding to who said making a native app is a huge undertaking, because learning SwiftUI is fairly quick. That’s not to say that the maintainer should learn it but just that it’s fairly quick to learn should someone else want to.

I was also saying that someone (maybe someone other than the maintainer of DiffusionBee) could contribute a SwiftUI front end.

Finally I was saying I would gladly contribute it myself if I could (but unfortunately have other reasons why I can’t)

anyway hopefully that clears things up, and that hostility from your post is unwarranted.

Re: Stable Diffusion with Core ML on Apple Silicon

#100
post #80

Earlier quoted context omitted.

No part of it.

I mean, it is a legal time bomb in general[0], with a non-standard license that has special stipulations in an amendment. Do you really incur the weeks of lead time that it would take Legal to review the legality of redistributing this model? 0: https://github.com/CompVis/stable-diffusion/blob/main/LICENS...

Redistributing that model to end users that violate Attachment A seems like a minefield.
Post reply on HN