Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Run Stable Diffusion on Your M1 Mac’s GPU

replicate.com

Post reply on HN
Print view View original

361–370 of 415 posts

Re: Run Stable Diffusion on Your M1 Mac’s GPU

#361

Post by joshvm »

fomine3 wrote: post #274
joshvm wrote: post #225

Earlier quoted context omitted.

The latter. The major frameworks, at least, can be run in CPU-only mode, with a hardware abstraction layer for other devices (like CUDA-capable cards, TPUs etc). So practically it means you need an Nvidia GPU to get anywhere in a reasonable amount of time, but if you're not super dependent on latency (for inference) then CPU is an option. In principle, CPUs can run much bigger model inputs (at the expense of even mor…

Is there any blocker to have VRAM swap (on RAM or SSD)? It would make processing much slower, but it should be better than nothing (cause OOM) or alternatively run on CPU (more slower).

Not sure. I suspect the issue would be lots of memory transfer between the GPU and the CPU, because downstream layers usually need previous layer outputs. It would probably depend on the receptive field of the network? Also on how expensive memory transfer is, maybe it's worth it in some cases. But there's no reason why you couldn't run say the first big layers on the CPU and then treat deeper layers (which may take a smaller input) as a separate network to run on the GPU. I suppose you want the largest subgraph in your model that can fit in available VRAM. Certainly the Coral/EdgeTPU will dispatch unsupported operations to the CPU but that affects all ops beyond that point in the computation graph.

Re: Run Stable Diffusion on Your M1 Mac’s GPU

#362

Post by black3r »

forrestthewoods wrote: post #339
systemvoltage wrote: post #306

Earlier quoted context omitted.

I disagree, dependencies are sort of a universal problem. Ever had to set LD_LIBRARY_PATH? Python is pretty much innocent here.

> Python is pretty much innocent here. Could not possibly disagree more. https://xkcd.com/1987/ The problem with dependencies is that for very bad reasons people don’t ship them. Someone needs to package SD with a full copy of the Python runtime and every dependency. This should be the default method of distribution. #ShipYourDamnDependencies

Just don't mess up your system. The tutorial linked works perfectly fine in a clean python environment.

That XKCD is more about messing up your system by not knowing what you're doing but randomly following shitty tutorials which suggest stuff that collide with each other... Python's only fault in this is that it's a simple language and thus attracts people who aren't software engineers (students and math majors) and thus mostly don't know or care how to keep your system clean but love writing tutorials.

It's pretty easy to keep your pythons clean, don't use conda, never run pip with sudo, never run pip with --user..., never run pip outside a virtualenv (a good safety measure for that is to have pip point to nothing in your user shell, you can access system python with python3/pip3 if needed)... To check your python is clean, create a new environment and run pip freeze, it should output nothing.

pyenv is a non-destructive system for managing multiple pythons and virtualenvs (all pythons and envs get installed into ~/.pyenv), pip is a good system for distributing dependencies (when library authors don't skip out on providing binary wheels, and software authors use pip freeze to generate requirements.txt files).

Re: Run Stable Diffusion on Your M1 Mac’s GPU

#363

Post by hhjinks »

jw1224 wrote: post #76
usehackernews wrote: post #68

Magnusviri[0], the original author of the SD M1 repo credited in this article, has merged his fork into the Lstein Stable Diffusion fork. You can now run the Lstein fork[1] with M1 as of a few hours ago. This adds a ton of functionality - GUI, Upscaling & Facial improvements, weighted subprompts etc. This has been a big undertaking over the last few days, and I highly recommend checking it out. See the mac m1 readme…

Brilliant, thank you! I just got OP's setup working, but this seems much more user-friendly. Giving it a try now... EDIT: Got it working, with a couple of pre-requisite steps: 0. `rm` the existing `stable-diffusion` repo (assuming you followed OP's original setup) 1. Install `conda`, if you don't already have it: brew install --cask miniconda 2. Install the other build requirements referenced in OP's setup: brew inst…

I had to manually install pytorch for the preload_models.py step to work, because ReduceOp wasn't found. Why even use anaconda if all the dependencies aren't included? Every time I touch an ML project, there's always a python dependency issue. How can people use a tool that's impossible to provide a consistent environment for?

Re: Run Stable Diffusion on Your M1 Mac’s GPU

#364

Post by werdnapk »

rhacker wrote: post #70
joshmlewis wrote: post #39

Earlier quoted context omitted.

When the model detects NSFW content it replaces the output with the frame of Rick Astley.

It's kind of amazing that ML can now intelligently rick roll people. I think it would be awesome to update the rickroll feature to the following: Auto Re-run the img2img with some text prompt: "all of the people are now Rick Astley" with low strength so it can adjust the faces, but not change the nudity!!!1

To be fair, the developers added this "feature" and can easily be disabled in the code. The ML just says "this might be NSFW".

Re: Run Stable Diffusion on Your M1 Mac’s GPU

#365

Post by slumpa »

Tenoke wrote: post #355
rattray wrote: post #350

Earlier quoted context omitted.

JS isn't perfect, but it's so much easier to deal with than Python in these regards.

Js libs don't need to care for things like system packages and drivers as much as Python ML does.

Ever tried running JS libs with C bindings? You're going to run into the exact same problems as you have running python code with C bindings.

Re: Run Stable Diffusion on Your M1 Mac’s GPU

#366

Post by wut42 »

Myrmornis wrote: post #356

The various articles/tutorials seem a bit confusing: even though they say "M1", they also worked fine for me on an Intel Mac (and does end up using GPU). Does anyone know how to think about the --W --H and --f flags to create larger images? I have 64GB memory, but I get errors from PyTorch saying things like "Invalid buffer size: 7.54 GB" when I try to increase W and H, and I haven't managed to make the Python proces…

That must be using CUDA then, and you need a gpu with at least 8GB of VRAM, afaik (not RAM).

Re: Run Stable Diffusion on Your M1 Mac’s GPU

#367

Post by totetsu »

usehackernews wrote: post #68

Magnusviri[0], the original author of the SD M1 repo credited in this article, has merged his fork into the Lstein Stable Diffusion fork. You can now run the Lstein fork[1] with M1 as of a few hours ago. This adds a ton of functionality - GUI, Upscaling & Facial improvements, weighted subprompts etc. This has been a big undertaking over the last few days, and I highly recommend checking it out. See the mac m1 readme…

What do I need for the in painting? is there a source for the models/ldm/inpainting_big/last.ckpt' file?

Re: Run Stable Diffusion on Your M1 Mac’s GPU

#368

Post by yboris »

bfirsh wrote: post #78
gonehome wrote: post #33

Thanks for this - it's rare to see a setup guide that actually works on each step! I did need to run the troubleshooting step too, could probably just move that up as a required step in the guide.

It isn't required for some (most?) users. Weirdly sometimes pip is picking up the wheel for `onnx`, sometimes it isn't, and we can't figure out why. Any Python packaging experts know what's going on? all macOS 12, arm64, Python 3.10. Can't think it wouldn't resolve the wheel. But yes, good idea to move up. I'll stick it next to the `pip install`.

Please consider also adding a small note to help those few that get stuck with this bug:

    RuntimeError: expected scalar type BFloat16 but found Float
The solution is easy:

    append the execution command with `--precision full`

Re: Run Stable Diffusion on Your M1 Mac’s GPU

#369

Post by practice9 »

bfirsh wrote: post #235
sork_hn wrote: post #182

Earlier quoted context omitted.

Same output for me also. EDIT: https://github.com/lstein/stable-diffusion/issues/293#issuec... fixed it for me.

Boom - nice. Here's a fork with that: https://github.com/bfirsh/stable-diffusion/tree/lstein Requirements are "requirements-mac.txt" which'll need subbing in the guide. We're testing this out with a few people in Discord before shipping to the blog post.

Thank you for these guides!

Re: Run Stable Diffusion on Your M1 Mac’s GPU

#370

Post by gregsadetsky »

Retr0id wrote: post #287
grishka wrote: post #286

Earlier quoted context omitted.

> Also, interesting/curious small note: images generated with this script are "invisibly watermarked" i.e. steganographied! Why?

So that future iterations of StableDiffusion (or similar models) don't end up getting trained on their own outputs.

... so this means that watermarking an image you own is probably the only way to avoid it being used for training further models? :-)
Post reply on HN