here's a direct app store link, if anyone wants to try the iPhone app immediately: https://apps.apple.com/us/app/draw-things-ai-generation/id64...
congratulations to liuliu on the launch!
Stretch iPhone to its limit: 2GiB Stable Diffusion model runs locally on device
31–40 of 184 posts
Re: Stretch iPhone to its limit: 2GiB Stable Diffusion model runs locally on device
#32Hmmm, my ipad keeps crashing and redownloading the sd_v1.4_f16.ckpt file.
Re: Stretch iPhone to its limit: 2GiB Stable Diffusion model runs locally on device
#33I just tried it works beautifully!
I suggest defaulting to lower resolution 384 X 384 since it will speed up.
Re: Stretch iPhone to its limit: 2GiB Stable Diffusion model runs locally on device
#34Could in-memory compression used to bring down the RAM requirements? There are some performance compressors like Blosch tuned for this: https://www.blosc.org/pages/blosc-in-depth/ “Faster than memcpy” is the slogan.
MacOS has transparent memory compression. Unclear to me if that's made its way to iPhone, but if it hasn't yet it will sooner or later.
Memory compression is a generalization of swap, which is only for dynamic memory; files on disk don't need it because you can just read them off the disk.
The problem is that GPUs don't support virtual memory paging, so they can't read files nor decompress nor swap anything unless you write it yourself, which is a lot slower.
Also, ML models (probably) can't be compressed because they already are compressed; learning and compression are the same thing!
Re: Stretch iPhone to its limit: 2GiB Stable Diffusion model runs locally on device
#35Did you try using Core ML for inference?
Re: Stretch iPhone to its limit: 2GiB Stable Diffusion model runs locally on device
#36Earlier quoted context omitted.
MacOS has transparent memory compression. Unclear to me if that's made its way to iPhone, but if it hasn't yet it will sooner or later.
Memory compression? I can't find any good resources to read about it, any hints? I'm having trouble imaging how could it possibly work without totally destroying performance.
It doesn't destroy performance for the simple reason that nowadays memory access is slower than pure compute. If you need to use compute to produce some data to be stored in memory, your overall throughput could very well be faster than without compression.
There have been a large amount of innovation on fast compression and fast decompression in recent years. Traditional compression tools like gzip or xz are geared towards higher compression ratio, but memory compression tends to favor speed. Check out those algorithms:
* lz4: https://lz4.github.io/lz4/
* Google's snappy: https://github.com/google/snappy
* Facebook's zstd in fast mode: http://facebook.github.io/zstd/#benchmarks
Re: Stretch iPhone to its limit: 2GiB Stable Diffusion model runs locally on device
#37Hmmm, my ipad keeps crashing and redownloading the sd_v1.4_f16.ckpt file.
[deleted]
Re: Stretch iPhone to its limit: 2GiB Stable Diffusion model runs locally on device
#38This is really impressive !
Re: Stretch iPhone to its limit: 2GiB Stable Diffusion model runs locally on device
#39Hmmm, my ipad keeps crashing and redownloading the sd_v1.4_f16.ckpt file.
Yeah, seems iPad has bunch of issues (curiously most of it related to how I translate tensor back to CGImage ...) stay tuned.
Re: Stretch iPhone to its limit: 2GiB Stable Diffusion model runs locally on device
#40This is super cool. I just tried the default prompt on my iPhone 13 with the image size set to 768x512 and using the 3D Model (Redshift v1) and it just crashed the whole phone and restarted. Just like when I get BSOD's at work on my Windows GPU desktop :)