Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The unbearable cheapness of open weight models

jamesoclaire.com

Post reply on HN
Print view View original

131–140 of 195 posts

Re: The unbearable cheapness of open weight models

#131

Post by Onavo »

manwithopinions wrote: post #124
Tuna-Fish wrote: post #112

> What worries me about this is that Anthropic and OpenAI seem to have backed themselves into a corner of high costs. Can they reasonably decrease their prices by 20-50x to compete with DeepSeek or Xiaomi’s Mimo? They have high prices, not high costs. They will obviously keep prices as high as they can for as long as they can, while keeping demand up. Once demand starts to fall, so will the prices. > Are these models…

So why are they losing so much money? Money is made on the subset of inference that is charged at cost + margin via their APIs. API usage is so high because customers are still finding their feet, trying to understand how to measure the value they get from their spend, erring on the side of spend. Yes, in a world of unmeasured value and tokenmaxxing, inference is profitable on SOTA models because all capacity is bein…

They are overly bloated organizations too. The human costs are tremendously high. Good luck finding ML engineers paid 7-8 figs USD in Asia. Same quality engineers, different market.

Re: The unbearable cheapness of open weight models

#132

Post by aldonius »

an0malous wrote: post #105
beepdyboop wrote: post #87

I don’t get it. So many here are saying open weight models will kill the frontier labs. But open source and similar have tried to beat private companies everywhere all the time, and people still buy the best products even if great open source alternatives are available. Why wouldn’t this be the case for AI too?

The closest example I can think of is using a proprietary hosted database versus a self hosted open source option, like Oracle vs Postgres. OpenAI and Anthropic are each individually privately valued over $1T and Oracle is currently valued at half that. They’re not worthless, but they’re severely overvalued.

Oracle's not a good comparison though, because it's getting valued as an AI compute provider as well as for its traditional business. At the end of 2021 Oracle's stock was about $100 (all-time-high in nominal terms at the time), its current ATH was October last year at $292.

Re: The unbearable cheapness of open weight models

#133

Post by uejfiweun »

Tuna-Fish wrote: post #112

> What worries me about this is that Anthropic and OpenAI seem to have backed themselves into a corner of high costs. Can they reasonably decrease their prices by 20-50x to compete with DeepSeek or Xiaomi’s Mimo? They have high prices, not high costs. They will obviously keep prices as high as they can for as long as they can, while keeping demand up. Once demand starts to fall, so will the prices. > Are these models…

What are you even talking about? Everyone knows that Anthropic is drastically subsidizing their plans. It's actually the exact opposite of what you're talking about. The costs are extremely high and the prices are actually what's being subsidized and cheap right now.

Re: The unbearable cheapness of open weight models

#134

Post by Tuna-Fish »

manwithopinions wrote: post #124
Tuna-Fish wrote: post #112

> What worries me about this is that Anthropic and OpenAI seem to have backed themselves into a corner of high costs. Can they reasonably decrease their prices by 20-50x to compete with DeepSeek or Xiaomi’s Mimo? They have high prices, not high costs. They will obviously keep prices as high as they can for as long as they can, while keeping demand up. Once demand starts to fall, so will the prices. > Are these models…

So why are they losing so much money? Money is made on the subset of inference that is charged at cost + margin via their APIs. API usage is so high because customers are still finding their feet, trying to understand how to measure the value they get from their spend, erring on the side of spend. Yes, in a world of unmeasured value and tokenmaxxing, inference is profitable on SOTA models because all capacity is bein…

> So why are they losing so much money?

Mostly training. Claude didn't just get to be so good at coding by magic, it was suddenly so good because they did truly staggering amounts of RLHF and RLAIF on it. They are still doing that today, on any tasks they can figure out how to evaluate it on. This is capex for them.

Their margins on inference are >90% today for tokens they sell (plans are hard to count, but still profitable). Based on what we know of it's size and architecture, running Opus is not more than 2x more expensive than running Deepseek v4 pro, for which tokens are available at under 10% of the cost of Opus. Again, the reason their margins are 50% is because they are spending so much on things that are not inference, not because inference is expensive.

> The cheap model providers have a much better chance of achieving that.

Anthropic can do it with a push of a button, once they calculate that it will provide them better profit than current pricing.

Re: The unbearable cheapness of open weight models

#135

Post by Tuna-Fish »

uejfiweun wrote: post #133
Tuna-Fish wrote: post #112

> What worries me about this is that Anthropic and OpenAI seem to have backed themselves into a corner of high costs. Can they reasonably decrease their prices by 20-50x to compete with DeepSeek or Xiaomi’s Mimo? They have high prices, not high costs. They will obviously keep prices as high as they can for as long as they can, while keeping demand up. Once demand starts to fall, so will the prices. > Are these models…

What are you even talking about? Everyone knows that Anthropic is drastically subsidizing their plans. It's actually the exact opposite of what you're talking about. The costs are extremely high and the prices are actually what's being subsidized and cheap right now.

This is an example of common knowledge that is wrong. People look at their cash burn, assume that they spend this to subsidize inference, and get bonkers answers. Inference is not their largest expense.

Inference is cheap. Anthropic is only drastically subsidizing their plans if you count their training expenses as part of their costs.

Re: The unbearable cheapness of open weight models

#137

Post by CSSer »

Tuna-Fish wrote: post #134
manwithopinions wrote: post #124

Earlier quoted context omitted.

So why are they losing so much money? Money is made on the subset of inference that is charged at cost + margin via their APIs. API usage is so high because customers are still finding their feet, trying to understand how to measure the value they get from their spend, erring on the side of spend. Yes, in a world of unmeasured value and tokenmaxxing, inference is profitable on SOTA models because all capacity is bein…

> So why are they losing so much money? Mostly training. Claude didn't just get to be so good at coding by magic, it was suddenly so good because they did truly staggering amounts of RLHF and RLAIF on it. They are still doing that today, on any tasks they can figure out how to evaluate it on. This is capex for them. Their margins on inference are >90% today for tokens they sell (plans are hard to count, but still pro…

I think people miss this because these companies exist in a space that is new in tech, and that means lots of competition through PR and marketing. When that happens, it’s easy to feel like a company is telling you about everything they’ve been working on or are openly talking about what gives them their edge when in fact the opposite is often true.

Re: The unbearable cheapness of open weight models

#138

Post by 05 »

dominotw wrote: post #99
drudolph914 wrote: post #89

Earlier quoted context omitted.

I feel like this comment is just engagement farming, but I'll bite anyways there is a larger appetite for something like open source AI mostly b/c of price. we all know these labs have not figured out their pricing model, and we're all holding our breath out of fear of what the prices could be. also, if you consider that the only toll to knowledge work before was personal time, and now you need to pay $100s month jus…

ppl buy iphones over cheap android phones . android phones can do everything that an iphone does ( and better)

Which cheap (or expensive) phone does better log/raw video than iPhone 17 pro max?

Re: The unbearable cheapness of open weight models

#139

Post by manwithopinions »

Tuna-Fish wrote: post #134
manwithopinions wrote: post #124

Earlier quoted context omitted.

So why are they losing so much money? Money is made on the subset of inference that is charged at cost + margin via their APIs. API usage is so high because customers are still finding their feet, trying to understand how to measure the value they get from their spend, erring on the side of spend. Yes, in a world of unmeasured value and tokenmaxxing, inference is profitable on SOTA models because all capacity is bein…

> So why are they losing so much money? Mostly training. Claude didn't just get to be so good at coding by magic, it was suddenly so good because they did truly staggering amounts of RLHF and RLAIF on it. They are still doing that today, on any tasks they can figure out how to evaluate it on. This is capex for them. Their margins on inference are >90% today for tokens they sell (plans are hard to count, but still pro…

> Their margins on inference are >90% today for tokens they sell (plans are hard to count, but still profitable).

That doesn’t make any sense, it doesn’t add up. Have you seen how much money they’re raising and burning? We know that training does not cost tens of billions.

Brockman said OpenAI expects to spend $50 billion on compute this year. OpenAI’s revenue run rate is less than $50 billion for this year! For 90% margins to be possible on inference, you are suggesting that less than $5 billion of that compute spend is inference and over $45 billion of that compute is training.

Anthropic have been desperately trying to juggle capacity by shaping user behavior through peak time usage limits because they are struggling with capacity for inference.

Plan based usage is widely acknowledged to be subsidized, you are probably the only person on earth suggesting that plans are profitable.

Re: The unbearable cheapness of open weight models

#140

Post by therobots927 »

Tuna-Fish wrote: post #135
uejfiweun wrote: post #133

Earlier quoted context omitted.

What are you even talking about? Everyone knows that Anthropic is drastically subsidizing their plans. It's actually the exact opposite of what you're talking about. The costs are extremely high and the prices are actually what's being subsidized and cheap right now.

This is an example of common knowledge that is wrong. People look at their cash burn, assume that they spend this to subsidize inference, and get bonkers answers. Inference is not their largest expense. Inference is cheap. Anthropic is only drastically subsidizing their plans if you count their training expenses as part of their costs.

Are you an anthropic insider or something? Because if you are you should delete this comment. If you aren’t then you don’t know what the hell you’re talking about.
Post reply on HN