Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Desert Ant Labs: local, fast models that run on device

desertant.com

Post reply on HN
Print view View original

21–30 of 102 posts

Re: Desert Ant Labs: local, fast models that run on device

#21

Post by TechSquidTV »

chadash wrote: post #14
1dom wrote: post #10

This is a cool way of approaching AI models. I'm a big fan of local LLMs, local specific models like this should be even more powerful. > Every model is free up to 100k monthly active devices. No tokens, no logins. I dunno about the business model though. Cloud LLM billing makes sense: you're getting another computer to do work with each request, and using their compute via their gateway that they bill you. These loc…

> If I'm happy with the weights you've given me, and I'm not using your compute for inference and not wanting or needing any updates off you, why should you continue getting money off me and my customers? I definitely understand the appeal of the desire to “buy it once”, but I think there are a few issues: - almost no software is static. Look at a package like python Requests and even though it does the simplest thin…

LLMs are static though.

Re: Desert Ant Labs: local, fast models that run on device

#22

Post by Aurornis »

sejje wrote: post #20
Aurornis wrote: post #18

Earlier quoted context omitted.

> If I'm happy with the weights you've given me, and I'm not using your compute for inference and not wanting or needing any updates off you, why should you continue getting money off me and my customers? Because they own the IP and they get to decide the terms of how it’s licensed. This is like a company taking open source software, saying they like the code the community has given them, and asking why they should c…

He's not arguing about their right to license it how they please. He's arguing that someone ought to choose to do it another way. You made up a position to argue against.

No, you’re arguing against the first part of my comment without reading the whole thing where I addressed the value proposition.

A 100K free device license is a generous gift to small companies. Past 100K, it’s more than fair to ask that companies contribute. That’s a significant operation at that point and the model has obviously provided value.

If the model doesn’t provide any value to a >100K device company then they should train their own or not use it.

I could see how people unfamiliar with per-device or per-unit licensing fees would be confused, but this is a common business model. Scaling payments with the customer’s business success is one of the more fair ways to align vendor and customer while also providing a nice way to give freely to smaller businesses like this.

It’s a good thing. It’s silly that a company giving away a license for 100K devices, which covers small and medium hardware operations, is being criticized for this of all things.

Re: Desert Ant Labs: local, fast models that run on device

#24

Post by [deleted] »

TechSquidTV wrote: post #21
chadash wrote: post #14

Earlier quoted context omitted.

> If I'm happy with the weights you've given me, and I'm not using your compute for inference and not wanting or needing any updates off you, why should you continue getting money off me and my customers? I definitely understand the appeal of the desire to “buy it once”, but I think there are a few issues: - almost no software is static. Look at a package like python Requests and even though it does the simplest thin…

LLMs are static though.

[deleted]

Re: Desert Ant Labs: local, fast models that run on device

#25

Post by pveugen »

sipjca wrote: post #5

at first i got very excited about a new fast transcription model (voz) but turns out its just parakeet v3 with some new inference code which is macOS/iOS specific

It's an ANE optimized version of Parakeet, with our own inference, which enabled us to push performance to about 300x realtime speed on an iPhone 16/17. Our next gen Voz model is trained from scratch and will be at least twice as fast. Android and other platforms will land soon.

Re: Desert Ant Labs: local, fast models that run on device

#28

Post by pveugen »

nullbio wrote: post #6

This is a cool idea. The most useful one for me would be something that can process pdf files into a json schema. Title and tag generation from a post would also be useful. I'm interested in web app though.

OCR on steroids. Our Schemer model will soon be available (free form text to structured JSON). Once that lands, we want to jump into image to JSON.

Re: Desert Ant Labs: local, fast models that run on device

#29

Post by pveugen »

ashenke wrote: post #7

A lot of the models would be useful in a web context, to improve on the CMS we're making for clients. But they look like most of them are iOS only, few have a node package or something other, and all the benchmark are running it on modern iPhones so I doubt it would be that fast on a 20$ VPS.

Just a few are iOS first (pure practical timing/sequencing). We plan to make all models available cross-platform in the coming weeks.

Re: Desert Ant Labs: local, fast models that run on device

#30

Post by serf »

Aurornis wrote: post #18
1dom wrote: post #10

This is a cool way of approaching AI models. I'm a big fan of local LLMs, local specific models like this should be even more powerful. > Every model is free up to 100k monthly active devices. No tokens, no logins. I dunno about the business model though. Cloud LLM billing makes sense: you're getting another computer to do work with each request, and using their compute via their gateway that they bill you. These loc…

> If I'm happy with the weights you've given me, and I'm not using your compute for inference and not wanting or needing any updates off you, why should you continue getting money off me and my customers? Because they own the IP and they get to decide the terms of how it’s licensed. This is like a company taking open source software, saying they like the code the community has given them, and asking why they should c…

>Because they own the IP and they get to decide the terms of how it’s licensed.

how'd that work out for the rest of IP owners that had their stuff used as training data for use in model creation?

was the material they trained on produced in-house?

Post reply on HN