Run Stable Diffusion on Your M1 Mac’s GPU • Hacker News Board

Re: Run Stable Diffusion on Your M1 Mac’s GPU

#151

Post by smoldesu » Thu, Sep 01, 2022, 7:16 PM UTC

pwinnski wrote: post #112

hbn wrote: post #84

Earlier quoted context omitted.

Is that the difference in graphics performance between the M1 and M1 Pro or did the other person do something wrong? 70-90 minutes seems nuts

I have the M1 8GB I mentioned in my first comment, and the M1 Pro 16GB I mentioned in my second component, side-by-side. However, the first one was running a Stable Diffusion branch from earlier in the week, so I replaced using the same instructions. The only difference now is the physical hardware. The thing to understand is that the 8GB M1 has 8GB. When I run txt2img.py, my Activity Monitor shows a Python process w…

Sounds entirely like a swap-constrained operation. You need ~8gb of VRAM to load the uncompressed model into memory, which obviously won't work well on a Macbook with 8gb of memory.

Re: Run Stable Diffusion on Your M1 Mac’s GPU

#152

Post by toinewx » Thu, Sep 01, 2022, 7:20 PM UTC

usehackernews wrote: post #68

Magnusviri[0], the original author of the SD M1 repo credited in this article, has merged his fork into the Lstein Stable Diffusion fork. You can now run the Lstein fork[1] with M1 as of a few hours ago. This adds a ton of functionality - GUI, Upscaling & Facial improvements, weighted subprompts etc. This has been a big undertaking over the last few days, and I highly recommend checking it out. See the mac m1 readme…

Everything works excepts it only generates black images,

did you run

python scripts/preload_models.py

python scripts/dream.py --full_precision ?

Re: Run Stable Diffusion on Your M1 Mac’s GPU

#153

Post by ece » Thu, Sep 01, 2022, 7:20 PM UTC

yoyohello13 wrote: post #31

Has anybody had success getting newer AMD cards working? ROCm support seems spotty at best, I have a 5700xt and I haven't had much luck getting it working.

I tried getting pytorch vulkan inference working with radv, it gives me a missing dtype error in vkformat. Fp16 or normal precision have the same error. I think it's some bf16 thing.

Re: Run Stable Diffusion on Your M1 Mac’s GPU

#154

Post by ralferoo » Thu, Sep 01, 2022, 7:20 PM UTC

hbn wrote: post #84

antihero wrote: post #75

Earlier quoted context omitted.

On an M1 Pro 16GB it is taking a couple minutes for each image.

Is that the difference in graphics performance between the M1 and M1 Pro or did the other person do something wrong? 70-90 minutes seems nuts

My first-gen M1 MacBook Air with 16GB takes just under 4 minutes per image. Running top while it's generating shows memory usage fluctuating between 10GB and 13GB, so if you're running on 8GB it's probably swapping a lot.

Re: Run Stable Diffusion on Your M1 Mac’s GPU

#155

Post by bfirsh » Thu, Sep 01, 2022, 7:21 PM UTC

valley_guy_12 wrote: post #111

gzer0 wrote: post #103

The difference between an M2 air (8gb/512gb) versus an M1 pro (16gb/1tb) is much more than I expected. * M1 pro (16gb/1tb) can run the model in around 3 minutes. * M2 air (8gb/512gb) takes ~60 minutes for the same model. I knew there would be some throttling due to the m2 air's fanless model, but I had no idea it would be a 20x difference (albeit, the m1 pro does have double the RAM. I don't have any other macbooks t…

That's probably due to swapping due to the 8GB of RAM. People who have run Stable Diffusion on M2 airs with 16 GB of RAM seem to get performance that is in line with their GPU core count.

Correct. We've been seeing 8GB is super slow, >=16GB is fast. We'll add that to the prerequisites.

Re: Run Stable Diffusion on Your M1 Mac’s GPU

#156

Post by toinewx » Thu, Sep 01, 2022, 7:22 PM UTC

jw1224 wrote: post #76

usehackernews wrote: post #68

Magnusviri[0], the original author of the SD M1 repo credited in this article, has merged his fork into the Lstein Stable Diffusion fork. You can now run the Lstein fork[1] with M1 as of a few hours ago. This adds a ton of functionality - GUI, Upscaling & Facial improvements, weighted subprompts etc. This has been a big undertaking over the last few days, and I highly recommend checking it out. See the mac m1 readme…

Brilliant, thank you! I just got OP's setup working, but this seems much more user-friendly. Giving it a try now... EDIT: Got it working, with a couple of pre-requisite steps: 0. `rm` the existing `stable-diffusion` repo (assuming you followed OP's original setup) 1. Install `conda`, if you don't already have it: brew install --cask miniconda 2. Install the other build requirements referenced in OP's setup: brew inst…

I only get black images.

Re: Run Stable Diffusion on Your M1 Mac’s GPU

#157

Post by addandsubtract » Thu, Sep 01, 2022, 7:26 PM UTC

joshstrange wrote: post #137

It's insane to me how fast this is moving. I jumped through a bunch of hoops 2-3 days ago to get this running on my M1 Mac's GPU and now it's way easier. I imagine we will have a nice GUI (I'm aware of the web-ui, I haven't set it up yet) packaged in an mac .app by the end of next week. Really cool stuff.

I hope this kickstarts some kind of M1 migration. There are so many ML projects I'd like to try, but they all depend on CUDA.

Re: Run Stable Diffusion on Your M1 Mac’s GPU

#158

Post by Razengan » Thu, Sep 01, 2022, 7:26 PM UTC

mark_l_watson wrote: post #23

Thanks for writing this up!! I enjoyed getting TensorFlow running with the M1, although a multi-headed model I was working on wouldn’t run. I just made my Dad’s 101 year old birthday card using OpenAI’s image generating service (he loved it) and when I get home from travel I will use your instructions in the linked article. Any advice for running Stable Diffusion locally vs. Colab Pro or Pro+? My M1 MacBook Pro only…

101 years! Congratulations!! Does he own a suspiciously plain gold ring by any chance?

Re: Run Stable Diffusion on Your M1 Mac’s GPU

#159

Post by yieldcrv » Thu, Sep 01, 2022, 7:27 PM UTC

usehackernews wrote: post #68

Magnusviri[0], the original author of the SD M1 repo credited in this article, has merged his fork into the Lstein Stable Diffusion fork. You can now run the Lstein fork[1] with M1 as of a few hours ago. This adds a ton of functionality - GUI, Upscaling & Facial improvements, weighted subprompts etc. This has been a big undertaking over the last few days, and I highly recommend checking it out. See the mac m1 readme…

are there benchmarks?

I was following the github issue and the CPU bound one was at 4-5 minutes, the MDS one was at 30 seconds, then 18 seconds, and people were still calling that slow.

What is it currently at now?

and I don't know what "fast" is, to compare

What are the Windows 10 with nice Nvidia chips w/ CUDA getting? Just curious whats comprehensive

Re: Run Stable Diffusion on Your M1 Mac’s GPU

#160

Post by joshstrange » Thu, Sep 01, 2022, 7:30 PM UTC

addandsubtract wrote: post #157

joshstrange wrote: post #137

It's insane to me how fast this is moving. I jumped through a bunch of hoops 2-3 days ago to get this running on my M1 Mac's GPU and now it's way easier. I imagine we will have a nice GUI (I'm aware of the web-ui, I haven't set it up yet) packaged in an mac .app by the end of next week. Really cool stuff.

I hope this kickstarts some kind of M1 migration. There are so many ML projects I'd like to try, but they all depend on CUDA.

Yep, I was just thinking the same thing. M1/M2 appears to be a huge untapped resource for ML stuff as this proves. I maxed out my MBP Max and this is probably the first time I'm actually fully using the GPU cores and it's pretty freaking cool. Creating landscapes or fictional characters (think D&D) is already super fun, I look forward to playing with img2img some more as well.