Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Run Stable Diffusion on Your M1 Mac’s GPU

replicate.com

Post reply on HN
Print view View original

261–270 of 415 posts

Re: Run Stable Diffusion on Your M1 Mac’s GPU

#261

Post by icedchai »

solarkraft wrote: post #260
usehackernews wrote: post #68

Magnusviri[0], the original author of the SD M1 repo credited in this article, has merged his fork into the Lstein Stable Diffusion fork. You can now run the Lstein fork[1] with M1 as of a few hours ago. This adds a ton of functionality - GUI, Upscaling & Facial improvements, weighted subprompts etc. This has been a big undertaking over the last few days, and I highly recommend checking it out. See the mac m1 readme…

I ran into: ImportError: cannot import name 'TypeAlias' from 'typing' (/opt/homebrew/Caskroom/miniconda/base/envs/ldm/lib/python3.9/typing.py)

I ran into this. You need Python 3.10. I had to edit environment-mac.yaml and set python==3.10.6 ...

Re: Run Stable Diffusion on Your M1 Mac’s GPU

#262

Post by Gigachad »

amilios wrote: post #183

How long does it take to generate a single image? Is it in the 30 min type range or a few mins? It's hypothetically "possible" to run e.g. OPT175B on a consumer GPU via Huggingface Accelerate, but in practice it takes like 30 mins to generate a single token.

I'm using a 2021 Macbook Pro with the base tier M1 Pro and it generates images in about 1 minutes per image.

Re: Run Stable Diffusion on Your M1 Mac’s GPU

#263

Post by fezfight »

addandsubtract wrote: post #157
joshstrange wrote: post #137

It's insane to me how fast this is moving. I jumped through a bunch of hoops 2-3 days ago to get this running on my M1 Mac's GPU and now it's way easier. I imagine we will have a nice GUI (I'm aware of the web-ui, I haven't set it up yet) packaged in an mac .app by the end of next week. Really cool stuff.

I hope this kickstarts some kind of M1 migration. There are so many ML projects I'd like to try, but they all depend on CUDA.

It's not really better to move from one closed eco system to another. We should collectively agree to strengthen more open platforms, shouldn't we?

Re: Run Stable Diffusion on Your M1 Mac’s GPU

#264

Post by wokwokwok »

r3trohack3r wrote: post #30

I've been playing with Stable Diffusion a lot the past few days on a Dell R620 CPU (24 cores, 96 GB of RAM). With a little fiddling (not knowing any python or anything about machine learning) I was able to get img2img.py working by simply comparing that script to the txt2img.py CPU patch. Was only a few lines of tweaking. img2img takes ~2 minutes to generate an image with 1 sample and 50 iterations, txt2img takes abo…

https://github.com/Birch-san/stable-diffusion has altered txt2img to support img2img and added other samplers, see:

https://github.com/Birch-san/stable-diffusion/blob/birch-mps...

That branch (birch-mps-waifu) runs on M1 macs no problem.

Re: Run Stable Diffusion on Your M1 Mac’s GPU

#265

Post by mark_l_watson »

Razengan wrote: post #158
mark_l_watson wrote: post #23

Thanks for writing this up!! I enjoyed getting TensorFlow running with the M1, although a multi-headed model I was working on wouldn’t run. I just made my Dad’s 101 year old birthday card using OpenAI’s image generating service (he loved it) and when I get home from travel I will use your instructions in the linked article. Any advice for running Stable Diffusion locally vs. Colab Pro or Pro+? My M1 MacBook Pro only…

101 years! Congratulations!! Does he own a suspiciously plain gold ring by any chance?

no :-)

Re: Run Stable Diffusion on Your M1 Mac’s GPU

#267

Post by alvb »

alvb wrote: post #239
sytse wrote: post #222

Earlier quoted context omitted.

I'm stuck on 'RuntimeError: expected scalar type BFloat16 but found Float' too. Most relevant links seems https://github.com/CompVis/stable-diffusion/pull/47 but I'm not sure. Please post when there is a solution.

That might have to do with your Mac OS version. Pre-12.4 Mac OS does not allow the Torch backend to use the M1 GPU, and so the script attempts to use the cpu, but then the cpu does not support half-precision numbers.

Yep---that was it in my case. I had the same error but it went away after upgrading to MacOS 12.5. You should actually check if your PyTorch installation can detect the mps backend: `torch.backends.mps.is_available()` must be equal to True.

Re: Run Stable Diffusion on Your M1 Mac’s GPU

#268

Post by bfirsh »

amelius wrote: post #4

I'd rather see someone implemented glue that allows you to run arbitrary (deep learning) code on any platform. I mean, are we going to see X on M1 Mac, for any X now in the future? Also, weren't torch and tensorflow supposed to be this glue?

As mentioned in sibling comments, Torch is indeed the glue in this implementation. Other glues are TVM[0] and ONNX[1]

These just cover the neural net though, and there is lots of surrounding code and pre-/post-processing that isn't covered by these systems.

For models on Replicate, we use Docker, packaged with Cog for this stuff.[2] Unfortunately Docker doesn't run natively on Mac, so if we want to use the Mac's GPU, we can't use Docker.

I wish there was a good container system for Mac. Even better if it were something that spanned both Mac and Linux. (Not as far-fetched as it seems... I used to work at Docker and spent a bit of time looking into this...)

[0] https://tvm.apache.org/ [1] https://onnx.ai/ [2] https://github.com/replicate/cog

Re: Run Stable Diffusion on Your M1 Mac’s GPU

#269

Post by itsuka »

solarkraft wrote: post #260
usehackernews wrote: post #68

Magnusviri[0], the original author of the SD M1 repo credited in this article, has merged his fork into the Lstein Stable Diffusion fork. You can now run the Lstein fork[1] with M1 as of a few hours ago. This adds a ton of functionality - GUI, Upscaling & Facial improvements, weighted subprompts etc. This has been a big undertaking over the last few days, and I highly recommend checking it out. See the mac m1 readme…

I ran into: ImportError: cannot import name 'TypeAlias' from 'typing' (/opt/homebrew/Caskroom/miniconda/base/envs/ldm/lib/python3.9/typing.py)

I followed the conda instruction which uses Python 3.9 and ran into the same issue. The workaround is to import TypeAlias from typing_extensions:

stable-diffusion/src/k-diffusion/k_diffusion/sampling.py

(before)

  from typing import Optional, Callable, TypeAlias
(after)

  from typing import Optional, Callable
  from typing_extensions import TypeAlias
This issue is tracked in https://github.com/lstein/stable-diffusion/issues/302

Re: Run Stable Diffusion on Your M1 Mac’s GPU

#270

Post by mr_toad »

quitit wrote: post #226
wincy wrote: post #208

Earlier quoted context omitted.

An RTX 3090ti with 24GB of VRAM is widely available now that the crypto markets have crashed for $1150 or so. They were $2500 a year ago if you could find them.

Arguably the cost effective solution is to use cloud services, since we're talking just a few seconds difference (or you might be lucky like one HN reader who got allocated an A100 today.) But to play devil's advocate there are clear strengths available to the different platforms. PCs can readily upgrade into high end GPUs, but the compromise is that this becomes a requirement as basic GPUs don't feature enough VRAM…

> Arguably the cost effective solution is to use cloud services

And running it on a fresh setup might help with the ‘works on my machine’ type of bugs that are being reported.

Post reply on HN