Desert Ant Labs: local, fast models that run on device
1–10 of 102 posts
Re: Desert Ant Labs: local, fast models that run on device
#2[deleted]
Re: Desert Ant Labs: local, fast models that run on device
#3[dead]
Re: Desert Ant Labs: local, fast models that run on device
#4[dead]
Re: Desert Ant Labs: local, fast models that run on device
#5at first i got very excited about a new fast transcription model (voz) but turns out its just parakeet v3 with some new inference code which is macOS/iOS specific
Re: Desert Ant Labs: local, fast models that run on device
#6This is a cool idea. The most useful one for me would be something that can process pdf files into a json schema. Title and tag generation from a post would also be useful. I'm interested in web app though.
Re: Desert Ant Labs: local, fast models that run on device
#7A lot of the models would be useful in a web context, to improve on the CMS we're making for clients. But they look like most of them are iOS only, few have a node package or something other, and all the benchmark are running it on modern iPhones so I doubt it would be that fast on a 20$ VPS.
Re: Desert Ant Labs: local, fast models that run on device
#8Shiny layer of marketing and proprietary code on top of open models?
Voz is Parakeet 0.6B v3
Clear is DeepFilterNet 3
Ear is the language predictor from whisper-tiny
...
Re: Desert Ant Labs: local, fast models that run on device
#9I wonder why they only support Apple platforms, citing CoreML. Doesn't Android have a similar framework, ML Kit?
Re: Desert Ant Labs: local, fast models that run on device
#10This is a cool way of approaching AI models. I'm a big fan of local LLMs, local specific models like this should be even more powerful.
> Every model is free up to 100k monthly active devices. No tokens, no logins.
I dunno about the business model though. Cloud LLM billing makes sense: you're getting another computer to do work with each request, and using their compute via their gateway that they bill you.
These local models are like old school software. They're producing the weights, and then giving them to people. If I'm happy with the weights you've given me, and I'm not using your compute for inference and not wanting or needing any updates off you, why should you continue getting money off me and my customers?
The whole "but we need to keep it updated for your security" doesn't really work as well for software designed to run fully offline like these local models are.
I'm not saying they shouldn't get paid, but I guess I feel a personal sadness that it's less obvious how to successfully monetise such a sincerely useful and beneficial approach towards AI models.