Very interested to see where this kind of work goes for on-device video generation!
1-Bit Bonsai Image 4B Image Generation for Local Devices
21–30 of 214 posts
Re: 1-Bit Bonsai Image 4B Image Generation for Local Devices
#22Anyone could pickup the minimal hardware requirements for this? Like both RAM and Storage?
The white paper says "mean-active memory pressure down to 1.95 GB for 1-bit Bonsai Image 4B and 2.38 GB for
Ternary Bonsai Image 4B". Storage is on the linked page, and is about half that.
Re: 1-Bit Bonsai Image 4B Image Generation for Local Devices
#23I actually can’t wait for the future where I upgrade hardware in order to upgrade my ai as an alternative to an expensive subscription. There are many problems I want to work on which require billions of tokens. These are completely inaccessible without corporate project sponsorship at the moment. An asic generation machine which can pump out a few 10s of thousands of tokens per second at opus4.6 quality is more than…
I'm curious how hardware and power cost would stack up to subscription cost
Re: 1-Bit Bonsai Image 4B Image Generation for Local Devices
#24Does anyone ever get their stuff to actually work. Like actually load?
Re: 1-Bit Bonsai Image 4B Image Generation for Local Devices
#25Anyone could pickup the minimal hardware requirements for this? Like both RAM and Storage?
The white paper says "mean-active memory pressure down to 1.95 GB for 1-bit Bonsai Image 4B and 2.38 GB for Ternary Bonsai Image 4B". Storage is on the linked page, and is about half that.
That is very low, looks like it should run in base MacMini M4 with 16GB RAM. I understand it is not released yet? What sort of harness is necessary for this type of model? (I have only used coding agents through GH Copilot in VS Code, the JetBrains AI tool and Pi, this last one was sort of a pain to setup…)
Re: 1-Bit Bonsai Image 4B Image Generation for Local Devices
#26Couldn't try it because the demo app is iOS only and the web version just crashes my browser. The small model is impressive but if you front load a 1.8GB text encoder model, the savings aren't quite as useful.
I do wonder how these compare to existing image generation models. I've tried https://github.com/alichherawalla/off-grid-mobile-ai for a while but I find the image generation models rather lacking.
Re: 1-Bit Bonsai Image 4B Image Generation for Local Devices
#27Does anyone ever get their stuff to actually work. Like actually load?
The online demos require WebGPU so Firefox on mobilr and privacy enhanced browsers will break. WebGPU support on Linux and other open source systems is also trash, you can force it to work in Chrome but it won't be happy.
Re: 1-Bit Bonsai Image 4B Image Generation for Local Devices
#28what trade off would one need to clear to justify the hardware and the work to get this running locally as part of a broader system? It’s a lot of work setting up and maintaining a production harness/system on a local device. I don’t personally repeatedly generate images at a scale where using a lab’s app somehow burns all my tokens. I like the ideas of local ai but I don’t see widespread adoption of it happening in commercial or customer situations anytime soon no matter how little/good enough they get. Even Uber- token burn whiplash but I doubt their answer will be “run some of it local”. IT nightmare, I’d imagine.
Re: 1-Bit Bonsai Image 4B Image Generation for Local Devices
#29To our knowledge, Bonsai Image 4B is the first image model in its parameter class to run directly on an iPhone.
Isn't SD XL 3.5B? And the refiner model is even larger. Those can run on an iPhone 13 Pro.
Re: 1-Bit Bonsai Image 4B Image Generation for Local Devices
#30The text encoder is still 4-bit quantized.