Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: Off Grid – Run AI text, image gen, vision offline on your phone

github.com

Post reply on HN
Print view View original

41–50 of 71 posts

Re: Show HN: Off Grid – Run AI text, image gen, vision offline on your phone

#41

Post by jeroenhd »

noodletheworld wrote: post #28

Nice idea, but isnt this kind of daft? There are basically no useful models that run on phone hardware. > Results vary by model size and quantization. I bet they do. Look, if you cant run models on your desktop, theres no way in hell they run on your phone. The problem with all of these self hosting solutions is that the actual models you can run on them aren't any good. Not like, “chat gpt a year ago” not good. Like…

> The iphone 17 pro has 12GB of ram.

I'm surprised Apple is still cheaping out on RAM on their phones, especially with the effort they've been putting into running AI locally and all of their NPU marketing.

Re: Show HN: Off Grid – Run AI text, image gen, vision offline on your phone

#43

Post by ruthlesssavage »

Game-changer.Just downloaded it and wow,this is the future. It’s incredible to have this much power fully offline. No lag, no privacy concerns, just pure utility. I’ve been playing with the vision AI all morning and it hasn't crashed once. 10/10. I have also used other GitHub projects but this feels premium, faster and optimised. Thank you for this!

Re: Show HN: Off Grid – Run AI text, image gen, vision offline on your phone

#44

Post by K0balt »

noodletheworld wrote: post #28

Nice idea, but isnt this kind of daft? There are basically no useful models that run on phone hardware. > Results vary by model size and quantization. I bet they do. Look, if you cant run models on your desktop, theres no way in hell they run on your phone. The problem with all of these self hosting solutions is that the actual models you can run on them aren't any good. Not like, “chat gpt a year ago” not good. Like…

Yeah. The solution if you want to have your own AI is to put a box online or rent cloud inference, and access it over a browser or a phone app.

We have on-prem AI for my microgrid community, but it’s a nascent effort and we can only run <100b models. At least that size is extremely useful for most stuff, and we have a selection of models to choose from on openAI /ollama compatible API endpoints.

Re: Show HN: Off Grid – Run AI text, image gen, vision offline on your phone

#45

Post by ali_chherawalla »

ruthlesssavage wrote: post #43

Game-changer.Just downloaded it and wow,this is the future. It’s incredible to have this much power fully offline. No lag, no privacy concerns, just pure utility. I’ve been playing with the vision AI all morning and it hasn't crashed once. 10/10. I have also used other GitHub projects but this feels premium, faster and optimised. Thank you for this!

thank you, appreciate it!

Re: Show HN: Off Grid – Run AI text, image gen, vision offline on your phone

#47

Post by ali_chherawalla »

jeroenhd wrote: post #41
noodletheworld wrote: post #28

Nice idea, but isnt this kind of daft? There are basically no useful models that run on phone hardware. > Results vary by model size and quantization. I bet they do. Look, if you cant run models on your desktop, theres no way in hell they run on your phone. The problem with all of these self hosting solutions is that the actual models you can run on them aren't any good. Not like, “chat gpt a year ago” not good. Like…

> The iphone 17 pro has 12GB of ram. I'm surprised Apple is still cheaping out on RAM on their phones, especially with the effort they've been putting into running AI locally and all of their NPU marketing.

with the metal infra its actually quite good. Agreed you can't run really large models, but inference is very fast and TTFT is very low. It's a beautiful experience
Post reply on HN