Live data from Hacker News

Stretch iPhone to its limit: 2GiB Stable Diffusion model runs locally on device

liuliu.me

161–170 of 184 posts

Re: Stretch iPhone to its limit: 2GiB Stable Diffusion model runs locally on device

#161

On a related note, has anyone been able to utilize Apple silicon GPUs? Running CPU-only is incredibly slow (and sad, since i've got these Apple accelerators idle!)

Automatic1111's stable diffusion web gui works with apple silicon: https://github.com/AUTOMATIC1111/stable-diffusion-webui/wiki...

Re: Stretch iPhone to its limit: 2GiB Stable Diffusion model runs locally on device

#162
post #107

Awesome! Although I wish to see the intermediate denoised image instead of the progress bar. Just a suggestion.

This isn't recommended, the decoding takes as much time as processing next step. I learned it the hard way when I tried displaying the intermediate steps for debugging.

yeah, running the full decoder takes a while. though, since the "latent" is just 4 channels and pretty close to representing RGB, you can use a linear combination of latent channels and get a basic (grainy, low-res) preview image like this [0] without much trouble. I expect you could go further, and train a shallow conv-only decoder to get nicer preview results, but I'm not sure if anyone's bothered yet.

[0] https://github.com/madebyollin/maple-diffusion

Re: Stretch iPhone to its limit: 2GiB Stable Diffusion model runs locally on device

#163

It works extremely fast on an iPad Pro M1 (kind of expected, but it's _impressive_) although the app is built as "iPhone only", and strangely enough the iPad is cropping the upscaled iPhone app so the lower bar of image controls don't show at all, which is a pity

This is about to become this single most important app on my iPad Pro. It totally accelerates my workflow

Re: Stretch iPhone to its limit: 2GiB Stable Diffusion model runs locally on device

#165
post #31

here's a direct app store link, if anyone wants to try the iPhone app immediately: https://apps.apple.com/us/app/draw-things-ai-generation/id64... congratulations to liuliu on the launch!

Crashes on my 13 Mini, which is unexpected! Anything we need to do to prep the phones?

[deleted]

Re: Stretch iPhone to its limit: 2GiB Stable Diffusion model runs locally on device

#166
post #45

This is super cool. I just tried the default prompt on my iPhone 13 with the image size set to 768x512 and using the 3D Model (Redshift v1) and it just crashed the whole phone and restarted. Just like when I get BSOD's at work on my Windows GPU desktop :)

You should see a warning when selecting that size? 4GiB model cannot run at that resolution until someone implemented FlashAttention on Metal :)

I did, it's fully my fault, I just wanted to see what would happen.

Love the work, really great job!

Re: Stretch iPhone to its limit: 2GiB Stable Diffusion model runs locally on device

#167
post #15

Earlier quoted context omitted.

SE 3rd Gen has 4GiB RAM, therefore the app defaults to 384x384 size. This is about 1/2 computation of your normal run (512x512) and the original version uses PLMS sampler, which defaults to 50 steps, while this one uses the newer DPM++ 2M Karras sampler, that defaults to 30 steps. All in all, your M1 Pro MBP is still 4x of your SE 3rd Gen in raw performance (although my implementation should be faster than PyTorch at…

> although my implementation should be faster than PyTorch at about 2x on M1 chips Do you plan to make a macOS version of your app also? Hope you will :)

For what it's worth, you can decrease resolution and use the sampler mentioned on the pytorch versions. The AUTOMATIC web UI supports this, for instance.

I would also welcome the additional optimizations, however.

Re: Stretch iPhone to its limit: 2GiB Stable Diffusion model runs locally on device

#168
post #31

here's a direct app store link, if anyone wants to try the iPhone app immediately: https://apps.apple.com/us/app/draw-things-ai-generation/id64... congratulations to liuliu on the launch!

Crashes on my 13 Mini, which is unexpected! Anything we need to do to prep the phones?

It's running on my 13 Mini (so far).

I did have trouble closing the "adjustments" dialog (upper-right button) due to its close button being underneath the status bar, but found that I could just drag the dialog down to the bottom and it closed.

Re: Stretch iPhone to its limit: 2GiB Stable Diffusion model runs locally on device

#169

Earlier quoted context omitted.

Crashes on my 13 Mini, which is unexpected! Anything we need to do to prep the phones?

I think that probably is to be expected, unfortunately. The model is 2GB; per the article iOS will kill apps that use 2GB on 4GB RAM devices (and the 13 Mini is 4GB).

It works just fine on my iPhone 12 Mini, which has (basically) the same RAM, so it seems like it's something else. They're basically the same phone in general though, so I would be surprised if it was a hardware issue.

Re: Stretch iPhone to its limit: 2GiB Stable Diffusion model runs locally on device

#170

I wonder if any of these tricks would be applicable to make a version of Stable Diffusion which could run on the Steam Deck.

Apple's mobile processors (especially the M1) are waaay faster than a steam deck. Even with the optimizations, it would take like half an hour to run I bet.
Post reply on HN