Stable Diffusion with Core ML on Apple Silicon • Hacker News Board

Re: Stable Diffusion with Core ML on Apple Silicon

#121

Post by l33tman » Fri, Dec 02, 2022, 9:34 AM UTC

astrange wrote: post #46

qclibre22 wrote: post #13

Earlier quoted context omitted.

Also, can you not "upgrade" but still run new models?

You can do anything you want. SD2 wasn’t “neutered”, the piece of it from OpenAI that knew a lot of artist names but wasn’t reproduceable was replaced with a new one from Stability that doesn’t. You can fine-tune anything you want back in.

The training-set was nerfed really good as well, it wasn't just OpenCLIP that was replaced. They will successively re-admit more training data during the 2.x releases I guess.

Re: Stable Diffusion with Core ML on Apple Silicon

#122

Post by jeroenhd » Fri, Dec 02, 2022, 9:49 AM UTC

sorenjan wrote: post #55

How come you always have to install some version of pytorch or tensor flow to run these ml models? When I'm only doing inference shouldn't there be easier ways of doing that, with automatic hardware selection etc. Why aren't models distributed in a standard format like onnx, and inference on different platforms solved once per platform?

Most models seem to be distributed by/for researchers and industry professionals. Stable Diffusion is state of the art technology, for example.

People who can't get the models to work by themselves given the source code aren't the target audience. There are other projects, though, that do distribute quick and easy scripts and tools to run these models.

Apple stepping in to get Stable Diffusion working on their platform is probably an attempt to get people to take their ML hardware more seriously. I read this more like "look, ma, no CUDA!" than "Mac users can easily use SD now". This module seemed to be designed so that the upstream SD code can easily be ported back to macOS without special tricks.

Re: Stable Diffusion with Core ML on Apple Silicon

#123

Post by cloogshicer » Fri, Dec 02, 2022, 10:43 AM UTC

I think it's sad that Apple doesn't even give attribution to any of the authors. If you copy the Bibtex from this site, the Author field is just empty. Their names are also not mentioned anywhere on this site.

This site is purely a marketing effort.

Re: Stable Diffusion with Core ML on Apple Silicon

#124

Post by rvz » Fri, Dec 02, 2022, 10:58 AM UTC

cloogshicer wrote: post #123

I think it's sad that Apple doesn't even give attribution to any of the authors. If you copy the Bibtex from this site, the Author field is just empty. Their names are also not mentioned anywhere on this site. This site is purely a marketing effort.

> I think it's sad that Apple doesn't even give attribution to any of the authors.

Pretty much like Stable Diffusion and the grifters using it in general and they will never credit the artists and images that they stole to generate these images.

Re: Stable Diffusion with Core ML on Apple Silicon

#125

Post by ClumsyPilot » Fri, Dec 02, 2022, 11:02 AM UTC

rvz wrote: post #124

cloogshicer wrote: post #123

I think it's sad that Apple doesn't even give attribution to any of the authors. If you copy the Bibtex from this site, the Author field is just empty. Their names are also not mentioned anywhere on this site. This site is purely a marketing effort.

> I think it's sad that Apple doesn't even give attribution to any of the authors. Pretty much like Stable Diffusion and the grifters using it in general and they will never credit the artists and images that they stole to generate these images.

Do your point is that Apple and those grifters are equally reputable?

two wrongs don't make a right.

Re: Stable Diffusion with Core ML on Apple Silicon

#126

Post by jclardy » Fri, Dec 02, 2022, 11:11 AM UTC

Terretta wrote: post #83

ncr100 wrote: post #77

Earlier quoted context omitted.

Agreed. And the posted benchmarks for the M2 Macbook Air make me consider 'upgrading' to an Air.

That laptop feels like liquid power. It's uncanny. Macbook Airs (way back when) felt sluggish. The MBA M1 changed that, it was "fine". These M2s are unexpectedly responsive on an ongoing basis. The MacBook Pro M1 Max is great (would be fantastic except they lost a Thunderbolt port in favor of legacy HDMI and memory card jacks), but you expect that machine to be responsive, so it's less surprising. The Studio Ultra, t…

I'd give the M1 air more credit - I moved from a 2019 16" Pro to the Air and performance was nearly identical except for long running tasks (> 10 minutes.) So for mobile app builds, it was blazing fast. And in the meantime the intel machine was blaring fans after the first 30 seconds while the Air barely got warm.And then the real kicker was watching the battery on the intel machine visibly dropping a few percentage points, while the air sits at the same level the whole time.

I've since moved to the M2 air, and it is noticeably faster than M1, but it isn't the huge leap from last gen intel that the M1 was. But the hardware itself feels way better.

Re: Stable Diffusion with Core ML on Apple Silicon

#127

Post by noduerme » Fri, Dec 02, 2022, 11:32 AM UTC

Can anyone explain in relatively lay terms how Apple's neural cores differ from a GPU? If they can run stable diffusion so much faster, which normally runs on a GPU, why aren't they used to run shaders for AAA games?

Re: Stable Diffusion with Core ML on Apple Silicon

#128

Post by huggingmouth » Fri, Dec 02, 2022, 11:38 AM UTC

sofaygo wrote: post #29

behnamoh wrote: post #25

This may sound naive, but what are some use cases of running SD models locally? If the free/cheap options exist (like running SD on powerful servers), then what's the advantage of this new method?

> There are a number of reasons why on-device deployment of Stable Diffusion in an app is preferable to a server-based approach. First, the privacy of the end user is protected because any data the user provided as input to the model stays on the user's device. Second, after initial download, users don’t require an internet connection to use the model. Finally, locally deploying this model enables developers to reduc…

Stability! The main reason why I use it locally is because I don't want some random dev unilaterally deciding to change or "sunsetting" features I rely on.

Centralized services small and large are guilty of this and I'm sick of it.

Re: Stable Diffusion with Core ML on Apple Silicon

#129

Post by GeekyBear » Fri, Dec 02, 2022, 12:02 PM UTC

sorenjan wrote: post #55

How come you always have to install some version of pytorch or tensor flow to run these ml models? When I'm only doing inference shouldn't there be easier ways of doing that, with automatic hardware selection etc. Why aren't models distributed in a standard format like onnx, and inference on different platforms solved once per platform?

>How come you always have to install some version of pytorch or tensor flow to run these ml models?

The repo is aimed at developers and has two parts. The first adapts the ML model to run on Apple Silicon (CPU, GPU, Neural Engine), and the second allows you to easily add Stable Diffusion functionality to your own app.

If you just want an end user app, those already exist, but now it will be easier to make ones that take advantage of Apple's dedicated ML hardware as well as the CPU and GPU.

>This repository comprises:

    python_coreml_stable_diffusion, a Python package for converting PyTorch models to Core ML format and performing image generation with Hugging Face diffusers in Python

    StableDiffusion, a Swift package that developers can add to their Xcode projects as a dependency to deploy image generation capabilities in their apps. The Swift package relies on the Core ML model files generated by python_coreml_stable_diffusion

https://github.com/apple/ml-stable-diffusion

Re: Stable Diffusion with Core ML on Apple Silicon

#130

Post by astrange » Fri, Dec 02, 2022, 12:26 PM UTC

rvz wrote: post #124

cloogshicer wrote: post #123

I think it's sad that Apple doesn't even give attribution to any of the authors. If you copy the Bibtex from this site, the Author field is just empty. Their names are also not mentioned anywhere on this site. This site is purely a marketing effort.

> I think it's sad that Apple doesn't even give attribution to any of the authors. Pretty much like Stable Diffusion and the grifters using it in general and they will never credit the artists and images that they stole to generate these images.

This is sort of like if you learned English from reading a book and the author said they owned all your English sentences after that.

Of course you can see the original images (https://rom1504.github.io/clip-retrieval/), it was legal to collect them (they used robots.txt for consent just like Google Image Search) and it was legal to do this with them (but not using US legal principles since it's made in Germany).

"Crediting the artist" isn't a legal principle - it's more like some kind of social media standard which is enforced by random amateur artists yelling at you if you don't do it. It's both impossible (there are no original artists for a given output) and wouldn't do anything to help the main social issue (future artists having their jobs taken by AIs).