Please someone package all of this and the WebUI into an Electron app so common people can also hack on it!
Run Stable Diffusion on Your M1 Mac’s GPU
221–230 of 415 posts
Re: Run Stable Diffusion on Your M1 Mac’s GPU
#222Earlier quoted context omitted.
Confirming I'm stuck on the same error when running the tutorial-instructed python scripts/txt2img.py command RuntimeError: expected scalar type BFloat16 but found Float
Yes, me too! Please post here if you find a solution for all the other people that come and find this by commmand-F'ing this error
I'm stuck on 'RuntimeError: expected scalar type BFloat16 but found Float' too. Most relevant links seems https://github.com/CompVis/stable-diffusion/pull/47 but I'm not sure. Please post when there is a solution.
Re: Run Stable Diffusion on Your M1 Mac’s GPU
#223Please someone package all of this and the WebUI into an Electron app so common people can also hack on it!
There are several packages that provide web UIs, like this one for example: https://github.com/hlky/stable-diffusion-webui
It's not quite the ease of setup of an Electron app, but once setup it's pretty easy to use.
Re: Run Stable Diffusion on Your M1 Mac’s GPU
#224Are we being pranked? I just followed the steps but the image output from my prompt is just a single frame of Rick Astley... EDIT: It was a false-positive (honest!) on the NSFW filter. To disable it, edit txt2img.py around line 325. Comment this line out: x_checked_image, has_nsfw_concept = check_safety(x_samples_ddim) And replace it with: x_checked_image = x_samples_ddim
If you open up the script txt2img and img2img scripts, there is a content filter. If your prompt generated anything that gets detected as "inappropriate" the image is replaced with Rick Astley. Removing the censor should be pretty straightforward, just comment out those lines.
It bothers me that this isn't just configurable. Why would they not want to expose this as a feature?
Re: Run Stable Diffusion on Your M1 Mac’s GPU
#225Earlier quoted context omitted.
I hope this kickstarts some kind of M1 migration. There are so many ML projects I'd like to try, but they all depend on CUDA.
Do they depend on CUDA, or are they just much better tuned for NVIDIA cards? I thought the whole ML ecosystem was based on training models and then running them on frameworks, where model was sorta like data and the framework handles the hardware? (albeit with models that can be tweaked to run more efficiently on different hardware) (I don't really know the ecosystem so it is definitely possible that they are more cl…
The latter. The major frameworks, at least, can be run in CPU-only mode, with a hardware abstraction layer for other devices (like CUDA-capable cards, TPUs etc). So practically it means you need an Nvidia GPU to get anywhere in a reasonable amount of time, but if you're not super dependent on latency (for inference) then CPU is an option. In principle, CPUs can run much bigger model inputs (at the expense of even more latency) because RAM is an order of magnitude more available typically.
Re: Run Stable Diffusion on Your M1 Mac’s GPU
#226Earlier quoted context omitted.
Maybe, but I can buy a Mac, you just order one from Apple.
An RTX 3090ti with 24GB of VRAM is widely available now that the crypto markets have crashed for $1150 or so. They were $2500 a year ago if you could find them.
Arguably the cost effective solution is to use cloud services, since we're talking just a few seconds difference (or you might be lucky like one HN reader who got allocated an A100 today.)
But to play devil's advocate there are clear strengths available to the different platforms. PCs can readily upgrade into high end GPUs, but the compromise is that this becomes a requirement as basic GPUs don't feature enough VRAM and CPU-only mode is woeful.
On the mac side of things, the GPU is not going to be the latest and greatest, but the M-series features unified memory, so a relatively normal M-series mac is going to have the necessary hardware to load the models. Not the fastest (but still fast), and ready to go. (Also as it stands the M-series can offer additional pathways to optimisation.)
Re: Run Stable Diffusion on Your M1 Mac’s GPU
#227Earlier quoted context omitted.
I got stuck on this roadblock, couldn’t get CUDA to work on my Mac, was very confusing
(base) stable-diffusion git:(main) conda env create -f environment.yaml Collecting package metadata (repodata.json): done Solving environment: failed ResolvePackageNotFound: - cudatoolkit=11.3 oh i was following the github fork readme, there is a special macos blog post
link?
Re: Run Stable Diffusion on Your M1 Mac’s GPU
#228Re: Run Stable Diffusion on Your M1 Mac’s GPU
#229Thanks for writing this up!! I enjoyed getting TensorFlow running with the M1, although a multi-headed model I was working on wouldn’t run. I just made my Dad’s 101 year old birthday card using OpenAI’s image generating service (he loved it) and when I get home from travel I will use your instructions in the linked article. Any advice for running Stable Diffusion locally vs. Colab Pro or Pro+? My M1 MacBook Pro only…
From the comments here 8GB is not enough, it will swap a lot and take way more time than a 16GB MacBook.
Re: Run Stable Diffusion on Your M1 Mac’s GPU
#230Running into this error `RuntimeError: expected scalar type BFloat16 but found Float` when I run `txt2img.py`
SOLUTION - append the command with `--precision full`