Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Desert Ant Labs: local, fast models that run on device

desertant.com

Post reply on HN
Print view View original

91–100 of 107 posts

Re: Desert Ant Labs: local, fast models that run on device

#91

Post by Muromec »

1dom wrote: post #62
handfuloflight wrote: post #45

Earlier quoted context omitted.

Analogy doesn't apply to this case, as you are paying for the chair upfront but not paying for the model weights upfront.

The only reason my chair manufacturer would be entitled to the proceeds of my success whilst I was in their chair is if I hadn't already paid them for the work they put into making the chair for me. Even then, why would my success with their product have anything to do with how much they can charge me for their labour? What other things work like that outside of the modern tech world? Seriously, look around you, pick…

>The only reason my chair manufacturer would be entitled to the proceeds of my success whilst I was in their chair is if I hadn't already paid them for the work they put into making the chair for me

Unless you could lease a chair and limit your cost to some finite number (which is higher then upfront cost), because chairs are easy to get by. You could also buy very cheap chairs from China for example and rent them by the hour (or that's what I heard).

The actual reason to why is -- because they can make you do it.

>What other things work like that outside of the modern tech world?

taxes

Re: Desert Ant Labs: local, fast models that run on device

#92

Post by ironsmoke »

nullbio wrote: post #6

This is a cool idea. The most useful one for me would be something that can process pdf files into a json schema. Title and tag generation from a post would also be useful. I'm interested in web app though.

You should check out IBM's Docling.

Re: Desert Ant Labs: local, fast models that run on device

#94

Post by horsawlarway »

weitendorf wrote: post #79
horsawlarway wrote: post #67

Earlier quoted context omitted.

50k annually for a static set of weights is actually crazy, though. As in - I make software for a living and they can fuck off. Produce the item. Sell the item at a cost you find reasonable. Convince me to buy it again with an upgrade that's worth buying. Period. There's no "service" happening here. If anything, the proposed cost model just makes me incredibly unlikely to engage at all, because it means they're going…

They're not selling it to you/it doesn't come out of your salary? They're selling it to your employer. You're literally not even the customer for this product. There's no need to get angry that they're charging enterprise/vendor rates for a product. And it's free for up to 100k devices. If you are running anything on 100k devices you should be able to afford $0.50/device and. If you aren't then why do you care? It's…

they literally would be selling it to me.

I deploy browser extensions to millions of devices, and small local models that I can run in that space are incredibly appealing.

But the "per user" model is just especially difficult to justify here, and comes with a TON of fucking downsides like usage tracking, feature availability, and product structure (ex - is a trial user a user? how long? etc...)

---

And it's not even like they're justifying it with model download support or caching... they shove it entirely onto hugging face, or have you self serve.

As their literal target audience - I'd rather buy the weights for a fixed price (even a high one), and not let them influence my product design decisions with saas style pricing.

Re: Desert Ant Labs: local, fast models that run on device

#95

Post by pavlov »

1dom wrote: post #56
chadash wrote: post #14

Earlier quoted context omitted.

> If I'm happy with the weights you've given me, and I'm not using your compute for inference and not wanting or needing any updates off you, why should you continue getting money off me and my customers? I definitely understand the appeal of the desire to “buy it once”, but I think there are a few issues: - almost no software is static. Look at a package like python Requests and even though it does the simplest thin…

I think the issues you're describing are all modern internet software issues. I'd guess you're possibly a bit younger than me, or maybe forgotten how software worked before about 2010. - All compiled software is static. New versions are released. I'm happy to pay for new versions/features if I want/need them. But if the version I have is working offline, it doesn't need security updates, and if it does what I need, i…

Are you forgetting that the upfront price for software used to be a lot higher than it is today?

You’re talking about times before 2010 and an EA game, but that’s not the right comparison. These Desert Ant AI models are an SDK, a professional middleware library. Licensing that kind of software in the pre-2010 days used to cost tens of thousands of dollars.

In other words, the proper comparison here is the various libraries that EA licensed to make the game, not the game itself.

Re: Desert Ant Labs: local, fast models that run on device

#96

Post by sureMan6 »

DrScientist wrote: post #80
1dom wrote: post #10

This is a cool way of approaching AI models. I'm a big fan of local LLMs, local specific models like this should be even more powerful. > Every model is free up to 100k monthly active devices. No tokens, no logins. I dunno about the business model though. Cloud LLM billing makes sense: you're getting another computer to do work with each request, and using their compute via their gateway that they bill you. These loc…

As far as I can see there is no clear pricing once you go beyond the free tier - it's contact sales. This is actually a classic enterprise software approach - as there are no fixed per unit production costs to cover, the pricing is very elastic. ie each license is probably a negotiation - if you see your app for a one off fee then they want a one-off cut - if you sell a subscription they may want a cut of that instea…

What's worrying for them is nothing they're doing is impossible to be recreated, if you bake it in and you don't want to pay rent you can fine tune your own model

Re: Desert Ant Labs: local, fast models that run on device

#98

Post by eproxus »

Dwedit wrote: post #93

Testing out Tongue, it detected "馬鹿外人" as Chinese.

It does say this: "Certain by script: these characters belong to only one language, so the model never ran."

So I think it's some buggy code in their UI that even prevents the model from running.

Re: Desert Ant Labs: local, fast models that run on device

#99

Post by anigbrowl »

I like the concept and the development choices seem sensible even if they're not my favorites (although I think missing Python is a mistake). The text feels very LLM generated though, and I reflexively discount the value of anything presented with this writing style.

Re: Desert Ant Labs: local, fast models that run on device

#100

Post by momojo »

> The world ships more than a billion capable phones, tablets, and laptops a year, most with a chip built for exactly this work, paid for and idle most of the day. Run the model there and the economics flip: no per-call cost, no round-trip, and nothing leaves the device.

This. I run small models (>50MB) for bio-imaging/biotech applications, it feels like every README implies that you need a discrete GPU to get started. While some do, many, especially the most useful ones, do not. Sure it matters if you're also going to do fine-tuning, but I believe your typical user just wants to detect some nuclei and get some cell-body ratios.

The laptop on your desk won't be running Meta's SAM, but it has more than enough compute to crunch 100's of your H&E slides overnight.

Post reply on HN