51–60 of 102 posts
I definitely think there's a lot to be done with small models dedicated to specific tasks. I've always thought the REAL value is in having large models be able to easily build small models for custom tasks (which I know is kind of a thing), but perhaps just providing the small models directly is the more accessible approach. >accessible via one SDK for Swift, Kotlin, and JavaScript Lol well let me know when there's a…
Shiny layer of marketing and proprietary code on top of open models? Voz is Parakeet 0.6B v3 Clear is DeepFilterNet 3 Ear is the language predictor from whisper-tiny ...
This is a cool idea. The most useful one for me would be something that can process pdf files into a json schema. Title and tag generation from a post would also be useful. I'm interested in web app though.
OCR on steroids. Our Schemer model will soon be available (free form text to structured JSON). Once that lands, we want to jump into image to JSON.
Once image support drops this could be a cool addition to https://struktur.sh. Will it be on OpenRouter too or will all inference have to be self-managed? I’m thinking about server-side use cases, where budgets are low, so small models shine.
edit: ah I saw mainly iOS for now. But an integration would be possible on macOS then, right?
> opinionated on-device intelligence > Hate speech triage. On-device moderation that flags hateful, abusive and threatening text What could go wrong here?
I don't think you understood. It means these are specialized models. Their toxic model could be ideal for video game lobbies without investing a ton of money if you're an indie dev This also could be ideal if you want your child to play online to have auto-censorship
This is a cool way of approaching AI models. I'm a big fan of local LLMs, local specific models like this should be even more powerful. > Every model is free up to 100k monthly active devices. No tokens, no logins. I dunno about the business model though. Cloud LLM billing makes sense: you're getting another computer to do work with each request, and using their compute via their gateway that they bill you. These loc…
> If I'm happy with the weights you've given me, and I'm not using your compute for inference and not wanting or needing any updates off you, why should you continue getting money off me and my customers? I definitely understand the appeal of the desire to “buy it once”, but I think there are a few issues: - almost no software is static. Look at a package like python Requests and even though it does the simplest thin…
- All compiled software is static. New versions are released. I'm happy to pay for new versions/features if I want/need them. But if the version I have is working offline, it doesn't need security updates, and if it does what I need, it doesn't need new features. Same applies if I'm building software for others, if the library does what I need for my software for my customers and I need nothing more off the dev, why should I keep paying the dev?
- I'm struggling not be sarcastic here, sorry, but this might shock you to hear: most humans find the idea of being able to try whatever they want and get paid for it appealing. That's not really how money or life works though.
- I don't know how to respond to this point, it's a bit of a strawman. Yes, if the only other option is to pay $50k up front for a local model, then your point is valid.
As a kid, I spent literally hundreds of £ on the first Sims, and Sims expansion packs at £20 /pop. EA got loads of money off me, I got loads of new features I wanted. I moaned at the time, but on reflection, everyone was a winner. Sims is still going now (possibly as a subscription!)
Again, apologies for a bit of sass here (that's a me problem), but the number of people who seem to sincerely believe the points you've put forward is why I'm personally sad at the state of modern software economics.
This is a cool way of approaching AI models. I'm a big fan of local LLMs, local specific models like this should be even more powerful. > Every model is free up to 100k monthly active devices. No tokens, no logins. I dunno about the business model though. Cloud LLM billing makes sense: you're getting another computer to do work with each request, and using their compute via their gateway that they bill you. These loc…
How would a model where you pay per each model/weight update feel like? More similar to traditional software, where you'd get the current version, maybe even free/rebated version upgrade to the next, but then have to do another purchase to get the latest version? Or something similar to that? Feels like it'd be more aligned with how the business and users would produce and use it, it'd be something like: "We've done…
If neither of those are true, a subscription or recurring license or similar regular fee just feels like greed/robbery.
Earlier quoted context omitted.
> If I'm happy with the weights you've given me, and I'm not using your compute for inference and not wanting or needing any updates off you, why should you continue getting money off me and my customers? I definitely understand the appeal of the desire to “buy it once”, but I think there are a few issues: - almost no software is static. Look at a package like python Requests and even though it does the simplest thin…
I think the issues you're describing are all modern internet software issues. I'd guess you're possibly a bit younger than me, or maybe forgotten how software worked before about 2010. - All compiled software is static. New versions are released. I'm happy to pay for new versions/features if I want/need them. But if the version I have is working offline, it doesn't need security updates, and if it does what I need, i…
> I don't know how to respond to this point, it's a bit of a strawman.
Why is it a strawman? I can sell you software for $100 once or I can sell it to you for $10/month. I don't think anyone is suggesting that I'm gonna sell it for $10 once though.
In particular, Desert Ant is saying their software is free for 100k active devices per SDK. That's likely a lot more users than most peoples' side projects. Given that, I'm guessing that $50k (even annually) is not a crazy number (yes, obviously $50k is not what a consumer would ever pay).