Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Run Stable Diffusion on Your M1 Mac’s GPU

replicate.com

Post reply on HN
Print view View original

271–280 of 415 posts

Re: Run Stable Diffusion on Your M1 Mac’s GPU

#271

Post by js2 »

A few suggested changes to the instructions:

    /opt/homebrew/bin/python3 -m venv venv  # [1, 2]
    venv/bin/python -m pip install -r requirements.txt  # [3]
    venv/bin/python scripts/txt2img.py ...
1. Using /opt/homebrew/bin/python3 allows you to remove the suggestion about "You might need to reopen your console to make it work" and ensures folks are using the just installed via homebrew python3, as opposed to Apple's /usr/bin/python3 which is currently 3.8. It also works regardless of the user's PATH. We can be fairly confident /opt/homebrew/bin is correct since that's the standard homebrew location on Apple Silicon and folks who've installed it elsewhere will likely know how to modify the instructions.

2. No need to install virtualenv since Python 3.6 which ships with a built-in venv module which covers most use cases.

3. No need to source an activate script. Call the python inside the virtual environment and it will use the virtual environment's packages.

Re: Run Stable Diffusion on Your M1 Mac’s GPU

#272

Post by schappim »

I just got rick-rolled by the model.

Using the prompt: "1990s textbook background mephis style"[sic] (yup I meant memphis)[0], I got back this: [1]. Rerunning the same prompt, I got: [2].

[0] https://files.littlebird.com.au/Shared-Image-2022-09-02-10-2...

[1] https://files.littlebird.com.au/grid-0004-2xXAGF.png

[2] https://files.littlebird.com.au/grid-0005-kcfgq7.png

Re: Run Stable Diffusion on Your M1 Mac’s GPU

#273

Post by kenrose »

icedchai wrote: post #261
solarkraft wrote: post #260

Earlier quoted context omitted.

I ran into: ImportError: cannot import name 'TypeAlias' from 'typing' (/opt/homebrew/Caskroom/miniconda/base/envs/ldm/lib/python3.9/typing.py)

I ran into this. You need Python 3.10. I had to edit environment-mac.yaml and set python==3.10.6 ...

This worked for me too.

Re: Run Stable Diffusion on Your M1 Mac’s GPU

#274

Post by fomine3 »

joshvm wrote: post #225
bee_rider wrote: post #186

Earlier quoted context omitted.

Do they depend on CUDA, or are they just much better tuned for NVIDIA cards? I thought the whole ML ecosystem was based on training models and then running them on frameworks, where model was sorta like data and the framework handles the hardware? (albeit with models that can be tweaked to run more efficiently on different hardware) (I don't really know the ecosystem so it is definitely possible that they are more cl…

The latter. The major frameworks, at least, can be run in CPU-only mode, with a hardware abstraction layer for other devices (like CUDA-capable cards, TPUs etc). So practically it means you need an Nvidia GPU to get anywhere in a reasonable amount of time, but if you're not super dependent on latency (for inference) then CPU is an option. In principle, CPUs can run much bigger model inputs (at the expense of even mor…

Is there any blocker to have VRAM swap (on RAM or SSD)? It would make processing much slower, but it should be better than nothing (cause OOM) or alternatively run on CPU (more slower).

Re: Run Stable Diffusion on Your M1 Mac’s GPU

#275

Post by fomine3 »

sethhochberg wrote: post #214
wincy wrote: post #208

Earlier quoted context omitted.

An RTX 3090ti with 24GB of VRAM is widely available now that the crypto markets have crashed for $1150 or so. They were $2500 a year ago if you could find them.

A twist on the above comment: I _already own_ an M2 Mac, but I'm never gonna buy a high-end GPU to play around with this sort of tech. If the things people (who aren't gamers, crypto miners, or ML researchers) already own can be useful for some hobby-level work in the space, we'll see a lot more work and experimentation in the space. Its super exciting stuff.

To be fair now many people have gaming PC. Perhaps more than who have their own M1 Pro/Max.

Re: Run Stable Diffusion on Your M1 Mac’s GPU

#276

Post by personjerry »

solarkraft wrote: post #260
usehackernews wrote: post #68

Magnusviri[0], the original author of the SD M1 repo credited in this article, has merged his fork into the Lstein Stable Diffusion fork. You can now run the Lstein fork[1] with M1 as of a few hours ago. This adds a ton of functionality - GUI, Upscaling & Facial improvements, weighted subprompts etc. This has been a big undertaking over the last few days, and I highly recommend checking it out. See the mac m1 readme…

I ran into: ImportError: cannot import name 'TypeAlias' from 'typing' (/opt/homebrew/Caskroom/miniconda/base/envs/ldm/lib/python3.9/typing.py)

TypeAlias is only used once, you can open sampling.py and remove the import on line 10 and the usage on line 14:

  from typing import Optional, Callable

  from . import utils

  TensorOperator = Callable[[Tensor], Tensor]

Re: Run Stable Diffusion on Your M1 Mac’s GPU

#277

Post by runeb »

squeaky-clean wrote: post #172
yieldcrv wrote: post #159

Earlier quoted context omitted.

are there benchmarks? I was following the github issue and the CPU bound one was at 4-5 minutes, the MDS one was at 30 seconds, then 18 seconds, and people were still calling that slow. What is it currently at now? and I don't know what "fast" is, to compare What are the Windows 10 with nice Nvidia chips w/ CUDA getting? Just curious whats comprehensive

> What are the Windows 10 with nice Nvidia chips w/ CUDA getting? Are you referring to single iteration step times, or whole images? Because obviously it depends on the number of iteration steps used. Windows 10, RTX 2070 (laptop model), lstein repo. I get about 3.2 iter/sec. A 50 step 512x512 image takes me 15 seconds.

Wow, that is over twice as fast as my Windows 11, RTX 3080ti

Re: Run Stable Diffusion on Your M1 Mac’s GPU

#278

Post by squeaky-clean »

Aeolun wrote: post #246
squeaky-clean wrote: post #172

Earlier quoted context omitted.

> What are the Windows 10 with nice Nvidia chips w/ CUDA getting? Are you referring to single iteration step times, or whole images? Because obviously it depends on the number of iteration steps used. Windows 10, RTX 2070 (laptop model), lstein repo. I get about 3.2 iter/sec. A 50 step 512x512 image takes me 15 seconds.

Huh, that’s the same speed I get on Collab. Pretty good.

I only run 1 sample at a time (batch size 1), forgot to mention that, and that affects the step time.

It looks like each additional image in a batch is cheaper than the 1st image. For example if I reduce my resolution so I can generate more in a single batch

1 image, 50 steps, 320x320: 5s

2 images, 50 steps, 320x320: 8s

3 images, 50 steps, 320x320: 11s

4 images, 50 steps, 320x320: 14s

And the trend continues, and my reported iteration/sec goes down as well. It's not accounting for the fact that with steps=50 and batch size=4 it's actually running 200 steps, just in 4 parallel parts.

Re: Run Stable Diffusion on Your M1 Mac’s GPU

#279

Post by squeaky-clean »

runeb wrote: post #277
squeaky-clean wrote: post #172

Earlier quoted context omitted.

> What are the Windows 10 with nice Nvidia chips w/ CUDA getting? Are you referring to single iteration step times, or whole images? Because obviously it depends on the number of iteration steps used. Windows 10, RTX 2070 (laptop model), lstein repo. I get about 3.2 iter/sec. A 50 step 512x512 image takes me 15 seconds.

Wow, that is over twice as fast as my Windows 11, RTX 3080ti

I just commented on another sibling comment (too late to edit the first one), but I forgot to mention my batch size is only 1. I think most people use batch size 4, so basically multiply my time by your batch size for a real comparison.

Re: Run Stable Diffusion on Your M1 Mac’s GPU

#280

Post by [deleted] »

schappim wrote: post #272

I just got rick-rolled by the model. Using the prompt: "1990s textbook background mephis style"[sic] (yup I meant memphis)[0], I got back this: [1]. Rerunning the same prompt, I got: [2]. [0] https://files.littlebird.com.au/Shared-Image-2022-09-02-10-2... [1] https://files.littlebird.com.au/grid-0004-2xXAGF.png [2] https://files.littlebird.com.au/grid-0005-kcfgq7.png

[deleted]
Post reply on HN