[flagged]
Desert Ant Labs: local, fast models that run on device
101–110 of 112 posts
Re: Desert Ant Labs: local, fast models that run on device
#102at first i got very excited about a new fast transcription model (voz) but turns out its just parakeet v3 with some new inference code which is macOS/iOS specific
This seems to be a common theme with these "European sovereign AI" companies. Mostly built on top of other open source work and slightly adjusted, often to collect grant money (although I'm not sure about that last part in this case).
Re: Desert Ant Labs: local, fast models that run on device
#103at first i got very excited about a new fast transcription model (voz) but turns out its just parakeet v3 with some new inference code which is macOS/iOS specific
It's an ANE optimized version of Parakeet, with our own inference, which enabled us to push performance to about 300x realtime speed on an iPhone 16/17. Our next gen Voz model is trained from scratch and will be at least twice as fast. Android and other platforms will land soon.
I understand. And inference optimizations are great. It's just that it's not a new model, and that's what I was excited about. I work on cross-platform transcription inference and was hoping for something more than just a re-badged parakeet v3
Look forward to the next model
Re: Desert Ant Labs: local, fast models that run on device
#104This is a cool way of approaching AI models. I'm a big fan of local LLMs, local specific models like this should be even more powerful. > Every model is free up to 100k monthly active devices. No tokens, no logins. I dunno about the business model though. Cloud LLM billing makes sense: you're getting another computer to do work with each request, and using their compute via their gateway that they bill you. These loc…
> If I'm happy with the weights you've given me, and I'm not using your compute for inference and not wanting or needing any updates off you, why should you continue getting money off me and my customers?
Because that's the contract you agreed to?
If you don't like the contract they offer, don't take it. You can choose something else. It's really as simple as that.
Re: Desert Ant Labs: local, fast models that run on device
#105Earlier quoted context omitted.
How is it not obvious and fair that they are asking you to pay them when you see success (defined as >100K MAU)? How more aligned can you and them be besides this?
When I buy I chair I don't pay a share of my income to the furniture shop when I get rich. I would buy another chair at some point too, maybe fancier one (or the same). Because chairs are commodity. I do however pay taxes to the government based on my income because it keeps doing ongoing maintenance on everything . Everyone wants to be paid forever for something they produced once is some kind of a mind virus. Make…
A chair manufacturer should be free to offer a service whereby you pay a monthly percentage of your income for the use of a chair.
I don't think many chair-users would opt for it, but I see no reason to ban it. Some services already work this way, like accounting firms that specialise in doing self-employed people's taxes.
Re: Desert Ant Labs: local, fast models that run on device
#106I recently had to use dictation for a few weeks and I was pleasantly surprised that many of the apps (in use/vogue) did support models on my 2021 16GB M1 Pro mac (many of those even supported connecting to a remote or local model endpoint) and at the same time for any worthwhile STT enhancement the model size was hitting higher I would have comfortably wanted. Even though I don't necessarily need dictation any more I intend to keep a custom fully offline setup and try these models (not sure they support live/streaming STT).
If any of you are interested there are apps like https://github.com/altic-dev/FluidVoice (this one's a great app) and this https://sam-pop.github.io/WhisperDictation. The latter, even though it has just 7 stars right now, seems to be more "intuitive". I just hope they expose a way to "connect" to available models on the machine or remotely)
Re: Desert Ant Labs: local, fast models that run on device
#107[flagged]
Re: Desert Ant Labs: local, fast models that run on device
#108Earlier quoted context omitted.
The only reason my chair manufacturer would be entitled to the proceeds of my success whilst I was in their chair is if I hadn't already paid them for the work they put into making the chair for me. Even then, why would my success with their product have anything to do with how much they can charge me for their labour? What other things work like that outside of the modern tech world? Seriously, look around you, pick…
> look around you, pick a thing, imagine a world where the price you paid for that thing hinged on your personal individual success with that thing YCombinator. Unreal Engine. CryEngine. FMOD. Steam, App stores in general. Qt. + 90% of enterprise software that charges per seat / user. You could argue that most of these provide ongoing services, but that will also be the case if you license one of their models commerc…
[flagged]
Re: Desert Ant Labs: local, fast models that run on device
#109That’s fun! I’ve been looking for something tiny I could embed in a webapp. Not a full genius model, just something light which could enhance the product without requiring ongoing cost.
Most organisations give their users terrible hardware, so anything which requires 32GB of RAM or a MacBook Pro won’t work if it’s a government or large organisation.
Re: Desert Ant Labs: local, fast models that run on device
#110Earlier quoted context omitted.
How is it not obvious and fair that they are asking you to pay them when you see success (defined as >100K MAU)? How more aligned can you and them be besides this?
When I buy I chair I don't pay a share of my income to the furniture shop when I get rich. I would buy another chair at some point too, maybe fancier one (or the same). Because chairs are commodity. I do however pay taxes to the government based on my income because it keeps doing ongoing maintenance on everything . Everyone wants to be paid forever for something they produced once is some kind of a mind virus. Make…
You haven’t bought the chair, they’ve given it to you for free, but they expect you to pay if you use it in a hugely profitable endeavour.
That is very reasonable.