Yesssss I've been waiting for this!
Run Stable Diffusion on Your M1 Mac’s GPU
211–220 of 415 posts
Re: Run Stable Diffusion on Your M1 Mac’s GPU
#212The difference between an M2 air (8gb/512gb) versus an M1 pro (16gb/1tb) is much more than I expected. * M1 pro (16gb/1tb) can run the model in around 3 minutes. * M2 air (8gb/512gb) takes ~60 minutes for the same model. I knew there would be some throttling due to the m2 air's fanless model, but I had no idea it would be a 20x difference (albeit, the m1 pro does have double the RAM. I don't have any other macbooks t…
Unscientifically that puts the M1 Pro GPU at about 25% of the performance of a RTX 3080. Not too shabby... EDIT - this comment implies it's much faster: https://news.ycombinator.com/item?id=32679518 If that's correct then it's close to matching my 3080 (mobile).
It's likely that a significant fraction of the perf difference between Apple' GPUs and NVIDIA GPUs is due to NVIDIA's CUDA being high optimized, and Pytorch being tuned to work with CUDA.
If Pytorch's metal support improves and Apple's Metal drivers improve (big ifs), it's likely that Apple's GPUs will perform better relatively to NVIDIA than they currently do.
Re: Run Stable Diffusion on Your M1 Mac’s GPU
#213Earlier quoted context omitted.
are there benchmarks? I was following the github issue and the CPU bound one was at 4-5 minutes, the MDS one was at 30 seconds, then 18 seconds, and people were still calling that slow. What is it currently at now? and I don't know what "fast" is, to compare What are the Windows 10 with nice Nvidia chips w/ CUDA getting? Just curious whats comprehensive
> What are the Windows 10 with nice Nvidia chips w/ CUDA getting? Are you referring to single iteration step times, or whole images? Because obviously it depends on the number of iteration steps used. Windows 10, RTX 2070 (laptop model), lstein repo. I get about 3.2 iter/sec. A 50 step 512x512 image takes me 15 seconds.
I’m referring to there being a community effort to normalize performance metrics and results at all, with the M1 devices being in that list as well, so that we dont have to ask these questions to begin with
Are you aware of any wiki or table like that?
Re: Run Stable Diffusion on Your M1 Mac’s GPU
#214Earlier quoted context omitted.
Maybe, but I can buy a Mac, you just order one from Apple.
An RTX 3090ti with 24GB of VRAM is widely available now that the crypto markets have crashed for $1150 or so. They were $2500 a year ago if you could find them.
A twist on the above comment: I _already own_ an M2 Mac, but I'm never gonna buy a high-end GPU to play around with this sort of tech. If the things people (who aren't gamers, crypto miners, or ML researchers) already own can be useful for some hobby-level work in the space, we'll see a lot more work and experimentation in the space. Its super exciting stuff.
Re: Run Stable Diffusion on Your M1 Mac’s GPU
#215Earlier quoted context omitted.
When the model detects NSFW content it replaces the output with the frame of Rick Astley.
It's kind of amazing that ML can now intelligently rick roll people. I think it would be awesome to update the rickroll feature to the following: Auto Re-run the img2img with some text prompt: "all of the people are now Rick Astley" with low strength so it can adjust the faces, but not change the nudity!!!1
Hah, it would be hilarious if it generated all the nudity you wanted - but with Rick Astley's face on every naked person!
Re: Run Stable Diffusion on Your M1 Mac’s GPU
#216Earlier quoted context omitted.
I suspect that the M2 air is thrashing the disk pretty aggressively. Diffusion models rerun the same model once per step, so for a generation with 50 steps, you copy the entire model in and out of memory 50 times. That's going to kill performance.
It's only copied to VRAM once when implemented correctly.
M1 is a unified memory system and doesn't have VRAM.
Re: Run Stable Diffusion on Your M1 Mac’s GPU
#217Anyone know the largest possible image size > 512x512? I'm getting the following error when trying 1024x1024 with 64 GB RAM on M1 MAX: /opt/homebrew/Cellar/python@3.10/3.10.6_2/Frameworks/Python.framework/Versions/3.10/lib/python3.10/multiprocessing/resource_tracker.py:224: UserWarning: resource_tracker: There appear to be 1 leaked semaphore objects to clean up at shutdown warnings.warn('resource_tracker: There appea…
Supposedly Stable Diffusion was trained on 512 x 512 images, so it's not clear that it will work well for larger images even if you had the RAM.
To generate larger images, it is standard practice to generate 512 x 512 and then use a separate tool to upscale, and maybe a second separate tool to improve the face. The Windows versions of SD environments are starting to incorporate these additional tools, but the Apple Silicon versions of SD environments are lagging behind due to Pytorch metal limitations.... It'll hopefully sort itself out in the next few months.
Re: Run Stable Diffusion on Your M1 Mac’s GPU
#218I keep running into issues, even after installing Rust in my condo environment (using conda). Specifically the issue seems to be building wheels for `tokenizers`:
warning: build failed, waiting for other jobs to finish...
error: build failed
error: `cargo rustc --lib --message-format=json-render-diagnostics --manifest-path Cargo.toml --release -v --features pyo3/extension-module -- --crate-type cdylib -C 'link-args=-undefined dynamic_lookup -Wl,-install_name,@rpath/tokenizers.cpython-310-darwin.so'` failed with code 101
[end of output]
note: This error originates from a subprocess, and is likely not a problem with pip.
ERROR: Failed building wheel for tokenizers
Failed to build tokenizers
ERROR: Could not build wheels for tokenizers, which is required to install pyproject.toml-based projects
Re: Run Stable Diffusion on Your M1 Mac’s GPU
#219Magnusviri[0], the original author of the SD M1 repo credited in this article, has merged his fork into the Lstein Stable Diffusion fork. You can now run the Lstein fork[1] with M1 as of a few hours ago. This adds a ton of functionality - GUI, Upscaling & Facial improvements, weighted subprompts etc. This has been a big undertaking over the last few days, and I highly recommend checking it out. See the mac m1 readme…
Brilliant, thank you! I just got OP's setup working, but this seems much more user-friendly. Giving it a try now... EDIT: Got it working, with a couple of pre-requisite steps: 0. `rm` the existing `stable-diffusion` repo (assuming you followed OP's original setup) 1. Install `conda`, if you don't already have it: brew install --cask miniconda 2. Install the other build requirements referenced in OP's setup: brew inst…
[deleted]
Re: Run Stable Diffusion on Your M1 Mac’s GPU
#220Is there anyway to keep up with this stuff / beginners guide? I really want to play around with it but it's kinda confusing to me. I don't have an M1 Mac, I have an Intel one with an AMD GPU, not sure if i can run it? don't mind if it's a bit slow, or what is the best way of running it in the cloud? Anything that can product high res for free?
https://beta.dreamstudio.ai/dream
It's not free but I've played with it a lot over the last two days for around $10, generating the most complex photos I can (1024x1024, 150 steps, 9 images, etc)