51–60 of 71 posts
Reminds me a lot of https://github.com/google-ai-edge/gallery which is a proof-of-concept app by Google themselves for their AI libraries. However, your app supports more and larger models without having to manually import anything, which is very useful.
Is there something similar, but geared towards a Linux desktop / laptop? I suppose this would be relatively easy to adapt.
Decently performing models for day to day use cases use at least 6GB VRAM each, though, and even then they're not coming very close to what the cheapest AI websites offer.
Looks useful, though something went wrong doing NPU image generation on my phone. Reminds me a lot of https://github.com/google-ai-edge/gallery which is a proof-of-concept app by Google themselves for their AI libraries. However, your app supports more and larger models without having to manually import anything, which is very useful.
Markdown rendering would be great Letting me download a model, wait for it, then. tell me I can't run it safely in my ram feels like it could've told me at the beginning
Bar that nice! Was it annoying integrating with apples AI stuff?
Are there any nice TTS models? It could be handy to have them as well.
I like it, I have a couple minor things (android) Markdown rendering would be great Letting me download a model, wait for it, then. tell me I can't run it safely in my ram feels like it could've told me at the beginning Bar that nice! Was it annoying integrating with apples AI stuff?
Yeah it wasn't straight forward figuring out the speed + getting it to work for both Android and iOS. Fair bit if complexity. But I'm so happy I got it done.
I was looking for something like this. Are you open for contributions?
I'm still working on a few things, but let me get a backlog in place and push it so people know what the roadmap looks like
Nice idea, but isnt this kind of daft? There are basically no useful models that run on phone hardware. > Results vary by model size and quantization. I bet they do. Look, if you cant run models on your desktop, theres no way in hell they run on your phone. The problem with all of these self hosting solutions is that the actual models you can run on them aren't any good. Not like, “chat gpt a year ago” not good. Like…
I actually think you should give it a spin. IMO you don't need claude level performance for a lot of day to day tasks. Qwen3 8B, or even 4B quantized is actually quite good. Take a look at it. You can offload to the GPU as well so it should really help with speed. Theres a setting for it
No, it’s not.
Trust me, I don't write this from a position of vague hand waving.
Ive tried a lot of self hosted models at a lot of sizes; those small models are not good enough, and do not have a context long enough to be useful for most everyday operations.