Live data from Hacker News

The unbearable cheapness of open weight models

jamesoclaire.com

191–195 of 195 posts

Re: The unbearable cheapness of open weight models

#191

Earlier quoted context omitted.

For one point, you can look at the costs of similarly sized open-source models from inference providers (which are only making money on the markup on the compute), and compare with anthropic's prices. There's a pretty big price difference there and it would be hard to believe that anthropic's models are that much more expensive to run than those models.

Those prices don't need to bake in the training cost, since it was eaten by someone else (whomever trained the open source models). Anthropic et al. need to price in the whole lifecycle.

Yes, but the point is specifically on whether inference in and of itself is profitable (i.e. whether the unit economics work out). They are still losing money overall, but they're not losing more money the more people use their product (quite the opposite: they need a lot of users at their fairly large per-token margins in order to justify their R&D spend, and the big question IMO is how strong the opposite side is: how much money do they need to spend on R&D to have a product that justifies such a high per-token premium?).

Re: The unbearable cheapness of open weight models

#192

Earlier quoted context omitted.

I feel like this comment is just engagement farming, but I'll bite anyways there is a larger appetite for something like open source AI mostly b/c of price. we all know these labs have not figured out their pricing model, and we're all holding our breath out of fear of what the prices could be. also, if you consider that the only toll to knowledge work before was personal time, and now you need to pay $100s month jus…

ppl buy iphones over cheap android phones . android phones can do everything that an iphone does ( and better)

there's still some lock-in here: data storage (iCloud), the broader Apple ecosystem, certain apps, user habits (I'm a lifelong Android user and am having trouble using other people's iPhones)

Re: The unbearable cheapness of open weight models

#193

Earlier quoted context omitted.

I don't think people's argument is that it's impossible to put data centers into space. The argument is that the downsides (radiation, cooling, maintenance, power) are so severe that it is pointless to do it at scale.

Go back to the megathreads when this came up. Even here on HN. Plenty of people used the argument that it can't be done, for various reasons. And my point was that at one point or the other there were many "downsides" for all the tech that SpaceX already has. Reusable boosters were seen as "uneconomical" and "pointless unless they can fly 10 times" by industry experts. They're now flying 30+times a booster. LEO const…

These guys aren't aware of all the "impossible" problems Elon already solved. They're too invested in the propaganda about him being a big dumb idiot who accidentally fell backwards into a pile of 1 trillion dollars.

Re: The unbearable cheapness of open weight models

#194
post #185
post #72

Earlier quoted context omitted.

https://aisle.com/blog/aisle-discovers-6-new-cves-in-curl-in...

I see "LLM discovers vulnerability in curl" and I get skeptical, given how Daniel Stenberg has talked about the flood of claimed vulnerabilities that weren't real issues once he looked into them (as most HN readers already know, I'm sure). But it looks like these 6 were real issues, that curl patched once they received the reports. Five ended up rated low and one medium, but given the amount of attention curl gets, I…

To be fair (and for people who didn't click the link), i think most of the vuln were in libcurl, not in curl itself.

Re: The unbearable cheapness of open weight models

#195

Earlier quoted context omitted.

So you can apply different weights to those "non-open" models? Also, I've read a bunch of descriptions of AI components, but none of them has said what the weights are applied to in the model. I guess that every model contains a dictionary of words and phrases, and the weights map relationships between them? All the descriptions simply talk about weights being applied to "input," but neglect to say what that input is…

The weights are just numbers. I don't what technical background you have in other areas of computing, but I think that this is a good, short introduction that doesn't assume too much: https://www.3blue1brown.com/lessons/mini-llm/ To quote part of it, Training a model can be thought of as tuning the dials on a really big machine. The way that a language model behaves is entirely determined by these many different cont…

Thanks for the reply. I've been programming professionally since the '90s (no background in ML or neural networks and whatnot), and I realize that the weights are numbers. So I was asking whether different weight sets can be applied to a model at will.

I'm really just trying to get a handle on the different layers so I can set up an environment and put it to work for limited coding assistance. For my purposes, I think an all-local setup is the best way for me to learn and should be enough for the coding tasks I have in mind. Mainly I want to automate tedious tasks away, like "modify these Swift classes' members and JSON deserialization routines to match what's coming out of my server API."

Thanks for the link! Reading it right now... and yes, this looks great. Appreciate it.

Post reply on HN