Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Stable Diffusion with Core ML on Apple Silicon

machinelearning.apple.com

Post reply on HN
Print view View original

101–110 of 184 posts

Re: Stable Diffusion with Core ML on Apple Silicon

#101

Post by ggerganov »

LoganDark wrote: post #98
sorenjan wrote: post #55

How come you always have to install some version of pytorch or tensor flow to run these ml models? When I'm only doing inference shouldn't there be easier ways of doing that, with automatic hardware selection etc. Why aren't models distributed in a standard format like onnx, and inference on different platforms solved once per platform?

Seconded, I wish for a way to work with ML models using native code rather than through some Python scripting interface. I believe TensorFlow is there with C++, but it works only with C++ and not through FFI.

It's one of the reasons I recently ported the Whisper model to plain C/C++. You just clone the repo, run `make [model]` and you are ready to go. No Python, no frameworks, no packages - plain and simple.

https://github.com/ggerganov/whisper.cpp

Re: Stable Diffusion with Core ML on Apple Silicon

#102

Post by peddling-brink »

password4321 wrote: post #78
zimpenfish wrote: post #43

Man, this takes a ton of room to do the CoreML conversions - ran out of space doing the unet conversion even though I started with 25GB free. Going on a delete spree to get it up to 50GB free before trying again.

All hail Grand Perspective back in the day, not sure who is carrying the "what's wasting my disk space" torch for free these days. Edit: still alive! https://grandperspectiv.sourceforge.net/

ncdu is the best in my book. TUI, supports deletion of files and folders, and very simple to understand.

GUI apps for this task like GP and the like are more visually complex than they need to be.

Re: Stable Diffusion with Core ML on Apple Silicon

#103

Post by kuwoze »

sorenjan wrote: post #55

How come you always have to install some version of pytorch or tensor flow to run these ml models? When I'm only doing inference shouldn't there be easier ways of doing that, with automatic hardware selection etc. Why aren't models distributed in a standard format like onnx, and inference on different platforms solved once per platform?

If you want it and it doesn't exist, why not simply do it yourself? It's open source no?

Re: Stable Diffusion with Core ML on Apple Silicon

#104

Post by microtonal »

LoganDark wrote: post #98
sorenjan wrote: post #55

How come you always have to install some version of pytorch or tensor flow to run these ml models? When I'm only doing inference shouldn't there be easier ways of doing that, with automatic hardware selection etc. Why aren't models distributed in a standard format like onnx, and inference on different platforms solved once per platform?

Seconded, I wish for a way to work with ML models using native code rather than through some Python scripting interface. I believe TensorFlow is there with C++, but it works only with C++ and not through FFI.

PyTorch has libtorch as its purely native library. There are also Rust bindings for libtorch:

https://github.com/LaurentMazare/tch-rs

I used this in the past to make a transformer-based syntax annotator. Fully in Rust, no Python required:

https://github.com/tensordot/syntaxdot

Re: Stable Diffusion with Core ML on Apple Silicon

#105

Post by microtonal »

modeless wrote: post #97
jerpint wrote: post #93

Earlier quoted context omitted.

How do dreamstudio/craiyon/hugging face manage to do seemingly quicker on their interfaces? Are they hosting these models on super beefy and costly GPUs for free?

M1's single-threaded CPU performance and power efficiency are exceptional; however M1's GPU performance is nothing special compared to normal discrete GPUs. You don't need something super beefy to beat M1 on the GPU side. But also yes, it's gotta be expensive to host these models and I'm not sure where all these subsidies are coming from. I expect that we'll eventually see these things transition to more paid service…

For a low-power SoC, the GPU performance is actually pretty impressive. We recently did some transformer benchmarks and the inference performance of the M1 Max is almost half that of an RTX3090:

https://explosion.ai/blog/metal-performance-shaders

However the SoC only uses 31W when posting that performance.

Re: Stable Diffusion with Core ML on Apple Silicon

#107

Post by Gigachad »

aryamaan wrote: post #33
artimaeis wrote: post #28

Earlier quoted context omitted.

A bit different take is DiffusionBee, if you're curious to try it out in a GUI form. https://diffusionbee.com

does it use the optimised model for Apple chips?

I just tested that app and it was taking about 1s/it using the "Double quality, double time" version. Spat out quite nice images at 25 iterations. Way better than stuff I had tried before which looked worse after a minute than this generates in 25 seconds.

Re: Stable Diffusion with Core ML on Apple Silicon

#108

Post by Gigachad »

behnamoh wrote: post #25

This may sound naive, but what are some use cases of running SD models locally? If the free/cheap options exist (like running SD on powerful servers), then what's the advantage of this new method?

You can set it to generate 100 images, hit start, come back later and scroll through the results. Can't do that without spending a bunch of money on the hosted services.

Re: Stable Diffusion with Core ML on Apple Silicon

#110

Post by serpix »

Terretta wrote: post #86
bredren wrote: post #45

Earlier quoted context omitted.

I’ve used this a fair amount but am not sure it’s much better place to begin than automatic1111, especially for the HN crowd.

automatic1111 does have an M1 workaround in the wiki, but it is incorrect it's correct enough that if you know your way around a CLI, git, and package management you can figure it out

It sucks to have to figure it out, any person who figures it out should submit a PR on the very outdated Apple Silicon readme.
Post reply on HN