This is absolutely incredible. It takes about 45 seconds to generate a whole image on my iPhone SE3 - which is about as fast as my M1 Pro macbook was doing it with the original version!
Stretch iPhone to its limit: 2GiB Stable Diffusion model runs locally on device
11–20 of 184 posts
Re: Stretch iPhone to its limit: 2GiB Stable Diffusion model runs locally on device
#12Earlier quoted context omitted.
MacOS has transparent memory compression. Unclear to me if that's made its way to iPhone, but if it hasn't yet it will sooner or later.
Memory compression? I can't find any good resources to read about it, any hints? I'm having trouble imaging how could it possibly work without totally destroying performance.
On Mac, you can find Compressed memory in Activity monitor.
It’s something similar to swap - apps do not need to have built in support for it.
Re: Stretch iPhone to its limit: 2GiB Stable Diffusion model runs locally on device
#13I'm downloading it now. This should be fun.
Re: Stretch iPhone to its limit: 2GiB Stable Diffusion model runs locally on device
#14Could in-memory compression used to bring down the RAM requirements? There are some performance compressors like Blosch tuned for this: https://www.blosc.org/pages/blosc-in-depth/ “Faster than memcpy” is the slogan.
MacOS has transparent memory compression. Unclear to me if that's made its way to iPhone, but if it hasn't yet it will sooner or later.
My guess is that iPhone is purely “kill app” instead of “compress memory / swap” OOM model. This makes more sense for mobile.
Re: Stretch iPhone to its limit: 2GiB Stable Diffusion model runs locally on device
#15This is absolutely incredible. It takes about 45 seconds to generate a whole image on my iPhone SE3 - which is about as fast as my M1 Pro macbook was doing it with the original version!
SE 3rd Gen has 4GiB RAM, therefore the app defaults to 384x384 size. This is about 1/2 computation of your normal run (512x512) and the original version uses PLMS sampler, which defaults to 50 steps, while this one uses the newer DPM++ 2M Karras sampler, that defaults to 30 steps. All in all, your M1 Pro MBP is still 4x of your SE 3rd Gen in raw performance (although my implementation should be faster than PyTorch at about 2x on M1 chips)
Re: Stretch iPhone to its limit: 2GiB Stable Diffusion model runs locally on device
#16works really well thank you!
Re: Stretch iPhone to its limit: 2GiB Stable Diffusion model runs locally on device
#17Earlier quoted context omitted.
Memory compression? I can't find any good resources to read about it, any hints? I'm having trouble imaging how could it possibly work without totally destroying performance.
On Mac, you can find Compressed memory in Activity monitor. It’s something similar to swap - apps do not need to have built in support for it.
It segments a certain amount of ram to “swap” to which means compress and store. Normal blue sky ram operations are not compressed on macOS
Re: Stretch iPhone to its limit: 2GiB Stable Diffusion model runs locally on device
#18Fantastic! Can it be made available for iPad?
Re: Stretch iPhone to its limit: 2GiB Stable Diffusion model runs locally on device
#19Earlier quoted context omitted.
Memory compression? I can't find any good resources to read about it, any hints? I'm having trouble imaging how could it possibly work without totally destroying performance.
On Mac, you can find Compressed memory in Activity monitor. It’s something similar to swap - apps do not need to have built in support for it.
Oh, yes compressed swap makes much more sense, thanks.
Re: Stretch iPhone to its limit: 2GiB Stable Diffusion model runs locally on device
#20Earlier quoted context omitted.
MacOS has transparent memory compression. Unclear to me if that's made its way to iPhone, but if it hasn't yet it will sooner or later.
My guess is that iPhone is purely “kill app” instead of “compress memory / swap” OOM model. This makes more sense for mobile.
IIRC from WWDC they said that inactive/suspended apps get their memory compressed to free up memory for the current active/foreground app.
Seems to mesh well with the iOS idea of using a single app at a time and minimizing background processing in apps that you aren't actively using.
In an out of memory situation I think apps just get killed as you suggest.