[flagged]
Desert Ant Labs: local, fast models that run on device
101–107 of 107 posts
Re: Desert Ant Labs: local, fast models that run on device
#102at first i got very excited about a new fast transcription model (voz) but turns out its just parakeet v3 with some new inference code which is macOS/iOS specific
This seems to be a common theme with these "European sovereign AI" companies. Mostly built on top of other open source work and slightly adjusted, often to collect grant money (although I'm not sure about that last part in this case).
Re: Desert Ant Labs: local, fast models that run on device
#103at first i got very excited about a new fast transcription model (voz) but turns out its just parakeet v3 with some new inference code which is macOS/iOS specific
It's an ANE optimized version of Parakeet, with our own inference, which enabled us to push performance to about 300x realtime speed on an iPhone 16/17. Our next gen Voz model is trained from scratch and will be at least twice as fast. Android and other platforms will land soon.
I understand. And inference optimizations are great. It's just that it's not a new model, and that's what I was excited about. I work on cross-platform transcription inference and was hoping for something more than just a re-badged parakeet v3
Look forward to the next model
Re: Desert Ant Labs: local, fast models that run on device
#104This is a cool way of approaching AI models. I'm a big fan of local LLMs, local specific models like this should be even more powerful. > Every model is free up to 100k monthly active devices. No tokens, no logins. I dunno about the business model though. Cloud LLM billing makes sense: you're getting another computer to do work with each request, and using their compute via their gateway that they bill you. These loc…
> If I'm happy with the weights you've given me, and I'm not using your compute for inference and not wanting or needing any updates off you, why should you continue getting money off me and my customers?
Because that's the contract you agreed to?
If you don't like the contract they offer, don't take it. You can choose something else. It's really as simple as that.
Re: Desert Ant Labs: local, fast models that run on device
#105Earlier quoted context omitted.
How is it not obvious and fair that they are asking you to pay them when you see success (defined as >100K MAU)? How more aligned can you and them be besides this?
When I buy I chair I don't pay a share of my income to the furniture shop when I get rich. I would buy another chair at some point too, maybe fancier one (or the same). Because chairs are commodity. I do however pay taxes to the government based on my income because it keeps doing ongoing maintenance on everything . Everyone wants to be paid forever for something they produced once is some kind of a mind virus. Make…
A chair manufacturer should be free to offer a service whereby you pay a monthly percentage of your income for the use of a chair.
I don't think many chair-users would opt for it, but I see no reason to ban it. Some services already work this way, like accounting firms that specialise in doing self-employed people's taxes.
Re: Desert Ant Labs: local, fast models that run on device
#106I recently had to use dictation for a few weeks and I was pleasantly surprised that many of the apps (in use/vogue) did support models on my 2021 16GB M1 Pro mac (many of those even supported connecting to a remote or local model endpoint) and at the same time for any worthwhile STT enhancement the model size was hitting higher I would have comfortably wanted. Even though I don't necessarily need dictation any more I intend to keep a custom fully offline setup and try these models (not sure they support live/streaming STT).
If any of you are interested there are apps like https://github.com/altic-dev/FluidVoice (this one's a great app) and this https://sam-pop.github.io/WhisperDictation. The latter, even though it has just 7 stars right now, seems to be more "intuitive". I just hope they expose a way to "connect" to available models on the machine or remotely)
Re: Desert Ant Labs: local, fast models that run on device
#107[flagged]