Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The unbearable cheapness of open weight models

jamesoclaire.com

Post reply on HN
Print view View original

111–120 of 195 posts

Re: The unbearable cheapness of open weight models

#111

Post by tuatoru »

beepdyboop wrote: post #87

I don’t get it. So many here are saying open weight models will kill the frontier labs. But open source and similar have tried to beat private companies everywhere all the time, and people still buy the best products even if great open source alternatives are available. Why wouldn’t this be the case for AI too?

Yes. Many industries are zero-sum-ish in nature,have winner-take-all dynamics or reputational costs for cheaping out. Financial trading. Big law. Military. National Security. Big insurance. Management consulting. Advertising.

For others even a small edge can be important. Pharma and Biochem research. Research in general. Any industry where there are major reputational risks.

It may not make sense to use the most expensive model to replace your payroll clerk, but there are plenty of use cases for the best available.

Re: The unbearable cheapness of open weight models

#112

Post by Tuna-Fish »

> What worries me about this is that Anthropic and OpenAI seem to have backed themselves into a corner of high costs. Can they reasonably decrease their prices by 20-50x to compete with DeepSeek or Xiaomi’s Mimo?

They have high prices, not high costs. They will obviously keep prices as high as they can for as long as they can, while keeping demand up. Once demand starts to fall, so will the prices.

> Are these models cheap because they are open weight and having hundreds or people stress test running them on different hardware helped to lower the cost? Or is it that they are being provided as loss leaders to drive the prices down?

Neither. They are cheap because they have neither technical edge nor brand power to keep the prices high, and so have to ask commodity prices for them.

People somehow still don't get it, despite everyone who studies the economics of it telling them: Inference is dirt cheap. Training is expensive, inference is cheap, and getting cheaper.

Re: The unbearable cheapness of open weight models

#113

Post by antonvs »

ForHackernews wrote: post #34
arikrahman wrote: post #6

With cache hit rates being effectively free, harnesses like Reasonix have let me do a month of work for less than 2 dollars. It's not even the subsidies making it cheap, American providers like Digital Ocean or Cloudflare host the same model with similar pricing.

I think this is very likely and something that everyone seems to be missing when valuing these AI firms. AI is not the new industrial revolution, it's the new cloud VM: a very useful commodity software offering.

The parallels to the Industrial Revolution are so close that we even have a new generation of Luddites. (Not saying they don’t have some valid points; so did the original group.)

The reason it’s like the Industrial Revolution is simply that there’s no question it’s going to completely transform jobs. It can make a very similar difference to the difference between a craftsman and a factory worker. The latter is massively more productive.

Re: The unbearable cheapness of open weight models

#114

Post by Schiendelman »

Tuna-Fish wrote: post #112

> What worries me about this is that Anthropic and OpenAI seem to have backed themselves into a corner of high costs. Can they reasonably decrease their prices by 20-50x to compete with DeepSeek or Xiaomi’s Mimo? They have high prices, not high costs. They will obviously keep prices as high as they can for as long as they can, while keeping demand up. Once demand starts to fall, so will the prices. > Are these models…

I get it! And I appreciate people like you pointing out the business side of LLMs.

Also, these open weight models are significantly lower quality than the high end coding models, and for some reason a lot of people think they're exactly the same. Maybe engineers who only dabble in LLM usage aren't doing enough complex work to notice...?

Re: The unbearable cheapness of open weight models

#115

Post by MoonWalk »

I'd appreciate an explanation of what "open weight model" means. Is it a "weight model" that is open, or a model with open weights (so should be "open-weight model"), or is it weights that can be applied to a model?

Are weights separable from a model? And if not, what is the point of saying "open-weight model" instead of just "open model?"

To the newcomer, it's hard to determine what the components of an AI system are from the throwing-around of these terms.

Re: The unbearable cheapness of open weight models

#116

Post by Schiendelman »

CuriouslyC wrote: post #96
Jackobrien wrote: post #7

The giants knew this was coming, and soon 95% of AI tasks will be able to be done by open models (coding, research, cowork style work). So why pay a premium? Why use them at all? This leaves the labs with two options: 1) push the frontier in a way only massive scale can, and cash in on it (mythos level cyber security, recursive training, frontier science work). There’s big money for never before possible capabilities…

#1 isn't going to happen because we're actually data limited, not compute limited. You can throw all the compute in the world at bad data and it won't make a difference, but an undertrained model with perfect training data will absolutely slay. #2 isn't going to happen, because these labs have shown they have limited app/design sense, and they also lack the industry connections and domain wisdom to execute. The way t…

I don't think "data limited" is true anymore outside of very specialized cases (for instance: https://arxiv.org/abs/2510.01631). As weird as it sounds, training improves a lot with synthetic data.

You do need business development to create those relationships. Saying they "have limited ___" mostly means they "haven't yet hired people who are good at ___". That's been changing already; the Claude app is steadily improving and handling more use cases simply through understanding which tools to use, Anthropic is building more relationships to create more tools, and all the frontier model companies are building relationships with companies that have specialized data and want specialized solutions.

I think we're also seeing the frontier model companies offer partners their own ability to run RL on their own data, and then retrain new models on the same data. That's going to make those relationships VERY sticky in ways that won't be obvious from the outside.

Re: The unbearable cheapness of open weight models

#117

Post by Zak »

1matin wrote: post #101
Zak wrote: post #95

One issue I keep seeing with cost comparisons is that they compare API rates while a substantial fraction of users are on subscription plans. It's more expensive to use GLM 5.2 paying z.ai or Opencode Zen API rates than it is to use Opus on a subscription plan. Both of those providers offer subscriptions priced favorably relative to their API rates, but only in what are effectively trial sizes.

And that means either: 1. They overprice their APIs to make their subscriptions look reasonable 2. They burn money with their subscriptions

Could be a little of each, plus a third option: subscription users don't always consume their entire quota.

Re: The unbearable cheapness of open weight models

#118

Post by philipkglass »

MoonWalk wrote: post #115

I'd appreciate an explanation of what "open weight model" means. Is it a "weight model" that is open, or a model with open weights (so should be "open-weight model"), or is it weights that can be applied to a model? Are weights separable from a model? And if not, what is the point of saying "open-weight model" instead of just "open model?" To the newcomer, it's hard to determine what the components of an AI system ar…

A completely open model is one like the Allen Institute's Olmo model series:

https://allenai.org/olmo

The trained weights are open, the training software is open, and the data that goes into training the model is open.

Not many models are fully open.

An open weights model is one that has freely available trained weights, and maybe fine-tuning tools, but it lacks the original training data (and usually lacks the training software). These are the most commonly used local models, like Google's Gemma series, Meta's Llama, or Alibaba's Qwen.

Re: The unbearable cheapness of open weight models

#119

Post by nomel »

CuriouslyC wrote: post #96
Jackobrien wrote: post #7

The giants knew this was coming, and soon 95% of AI tasks will be able to be done by open models (coding, research, cowork style work). So why pay a premium? Why use them at all? This leaves the labs with two options: 1) push the frontier in a way only massive scale can, and cash in on it (mythos level cyber security, recursive training, frontier science work). There’s big money for never before possible capabilities…

#1 isn't going to happen because we're actually data limited, not compute limited. You can throw all the compute in the world at bad data and it won't make a difference, but an undertrained model with perfect training data will absolutely slay. #2 isn't going to happen, because these labs have shown they have limited app/design sense, and they also lack the industry connections and domain wisdom to execute. The way t…

> we're actually data limited

Correction: public text data limited.

There's a ridiculous amount of proprietary text and non-text data out there that much of society is run on.

Re: The unbearable cheapness of open weight models

#120

Post by MoonWalk »

philipkglass wrote: post #118
MoonWalk wrote: post #115

I'd appreciate an explanation of what "open weight model" means. Is it a "weight model" that is open, or a model with open weights (so should be "open-weight model"), or is it weights that can be applied to a model? Are weights separable from a model? And if not, what is the point of saying "open-weight model" instead of just "open model?" To the newcomer, it's hard to determine what the components of an AI system ar…

A completely open model is one like the Allen Institute's Olmo model series: https://allenai.org/olmo The trained weights are open, the training software is open, and the data that goes into training the model is open. Not many models are fully open. An open weights model is one that has freely available trained weights, and maybe fine-tuning tools, but it lacks the original training data (and usually lacks the train…

So you can apply different weights to those "non-open" models?

Also, I've read a bunch of descriptions of AI components, but none of them has said what the weights are applied to in the model. I guess that every model contains a dictionary of words and phrases, and the weights map relationships between them?

All the descriptions simply talk about weights being applied to "input," but neglect to say what that input is compared to. If a user submits a query, are the words in the query weighed against the words in the model?

Can you recommend a primer on this whole process?

Post reply on HN