Live data from Hacker News

Stretch iPhone to its limit: 2GiB Stable Diffusion model runs locally on device

liuliu.me

101–110 of 184 posts

Re: Stretch iPhone to its limit: 2GiB Stable Diffusion model runs locally on device

#101
post #32

Hmmm, my ipad keeps crashing and redownloading the sd_v1.4_f16.ckpt file.

On my iPad mini 5th generation with A12 the download is fast and fine. But with standard settings it first warns “Device capability warning” and then indeed crashes every time. Is there a way to solve this? A12 chip should work, no?

Re: Stretch iPhone to its limit: 2GiB Stable Diffusion model runs locally on device

#102
post #80

Congrats on the release! I gave this and other available applications a try and I don’t understand what people see in ai image generation. A simple prompt generated a person with 3 nostrils, 7 squished fingers, deformities everywhere I look, it just mashes a bunch of photographs together and generates abominations. Pay close attention to generated models and you will find details which are simply wrong. What is the u…

I haven't been able to get any good results with Stable Diffusion (via DiffusionBee on my M1 MacBook Air), but I've seen really good images of other AI generators like Midjourney.

Re: Stretch iPhone to its limit: 2GiB Stable Diffusion model runs locally on device

#103
post #15
post #11

This is absolutely incredible. It takes about 45 seconds to generate a whole image on my iPhone SE3 - which is about as fast as my M1 Pro macbook was doing it with the original version!

SE 3rd Gen has 4GiB RAM, therefore the app defaults to 384x384 size. This is about 1/2 computation of your normal run (512x512) and the original version uses PLMS sampler, which defaults to 50 steps, while this one uses the newer DPM++ 2M Karras sampler, that defaults to 30 steps. All in all, your M1 Pro MBP is still 4x of your SE 3rd Gen in raw performance (although my implementation should be faster than PyTorch at…

> although my implementation should be faster than PyTorch at about 2x on M1 chips

Do you plan to make a macOS version of your app also? Hope you will :)

Re: Stretch iPhone to its limit: 2GiB Stable Diffusion model runs locally on device

#105
post #19

Earlier quoted context omitted.

On Mac, you can find Compressed memory in Activity monitor. It’s something similar to swap - apps do not need to have built in support for it.

Oh, yes compressed swap makes much more sense, thanks.

It is not compressed swap, the compressed data is still in RAM. The OS just compresses inactive memory, with a couple of criteria to define “inactive”.

Re: Stretch iPhone to its limit: 2GiB Stable Diffusion model runs locally on device

#108

This is cool and looks like a great way to drain my phone battery :) I just used the prompt "A person looking at their phone in amazement" and got a good picture. Beware that on startup the app downloads almost 2 gig of data.

I have generated some images, I think it only takes less than 1% of the battery for an image, this is already much better than most of the game(for having fun).

It took my battery from 80 to 77% for one generation on the default settings (384^2, 30 iterations). Less than a minute of compute time to complete a generation.

Iphone battery health reports a battery at 100% health. This is an iphone SE3.

Amazing how huge the difference in energy consumption is for the system in standby vs going full throttle.

EDIT: I generated 3 more images; every subsequent generation reduced battery capacity by another 2%. My phone doesn't seem to heat up at all, interestingly.

Re: Stretch iPhone to its limit: 2GiB Stable Diffusion model runs locally on device

#109
post #7

Earlier quoted context omitted.

MacOS has transparent memory compression. Unclear to me if that's made its way to iPhone, but if it hasn't yet it will sooner or later.

Memory compression is a generalization of swap, which is only for dynamic memory; files on disk don't need it because you can just read them off the disk. The problem is that GPUs don't support virtual memory paging, so they can't read files nor decompress nor swap anything unless you write it yourself, which is a lot slower. Also, ML models (probably) can't be compressed because they already are compressed; learning…

Wait. This comment just blew my mind. Does that imply that you might be able to measure the efficiency of a model by it's compressibility? Note, I'm trying to recognize efficient and accurate are not the same. One could imagine evaluating a model on a 2d performance and compression map somehow.
Post reply on HN