Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The unbearable cheapness of open weight models

jamesoclaire.com

Post reply on HN
Print view View original

171–180 of 195 posts

Re: The unbearable cheapness of open weight models

#171

Post by picofarad »

fredley wrote: post #23
Jackobrien wrote: post #7

The giants knew this was coming, and soon 95% of AI tasks will be able to be done by open models (coding, research, cowork style work). So why pay a premium? Why use them at all? This leaves the labs with two options: 1) push the frontier in a way only massive scale can, and cash in on it (mythos level cyber security, recursive training, frontier science work). There’s big money for never before possible capabilities…

3) Buy all the RAM, increasing the barrier to entry to push back the tide a bit, in time for a juicy IPO.

This is what it feels like they went with.

Re: The unbearable cheapness of open weight models

#172

Post by chairmansteve »

dada216 wrote: post #152
hakfoo wrote: post #141

Earlier quoted context omitted.

> Outside of coding, almost every business case for AI doesn’t need above human intelligence, it doesn’t even need human intelligence, or half a human intelligence, a business can extract a lot of value from a machine that has a fraction of a human’s intelligence. What the business world actually needed isn't intelligence, it's VBA with a bit of polish on it. Yeah, people want tools to distill reports, and puff nonse…

I have deployed very successful LLM based software that reads sales people emails and inserts orders, or stuff akin to orders, in the rest of the systems. Can you write me a regex that parses a messy human email thread and produces a clean JSON with all the order details? It's been working for a year with less than 3% error rate, better than what the humans themselves were doing.

Interesting. A web form completed by the sales people would not have sufficed? I guess it's the backend interfaces to multiple systems that's the difficult part?

Re: The unbearable cheapness of open weight models

#173

Post by amanaplanacanal »

Tuna-Fish wrote: post #112

> What worries me about this is that Anthropic and OpenAI seem to have backed themselves into a corner of high costs. Can they reasonably decrease their prices by 20-50x to compete with DeepSeek or Xiaomi’s Mimo? They have high prices, not high costs. They will obviously keep prices as high as they can for as long as they can, while keeping demand up. Once demand starts to fall, so will the prices. > Are these models…

You can't just define training as "not a cost". Without training they have nothing to sell.

Re: The unbearable cheapness of open weight models

#174

Post by holtkam2 »

Jimega36 wrote: post #8
odie5533 wrote: post #5

This is what concerns me about how AI giants are planning to make money. Their product has already been commoditized at prices which for them are still subsidized to grab market share. Unless the giants invent a technological leap, their prices are going to be dragged down by open weight models and I don't see how they'll turn a profit.

Reach AGI to leapfrog whoever is behind. Burn everything to get there faster.

If you used a time machine to go back to 2021 and showed someone the best open source LLMs from 2026 they would surely say “yeah that’s AGI”

Re: The unbearable cheapness of open weight models

#175

Post by julosflb »

amanaplanacanal wrote: post #155
rectang wrote: post #121

Earlier quoted context omitted.

License the training corpus and encourage copyright suits against outputs from models trained on unlicensed corpora.

This won't work if the courts decide that training is fair use, which certainly seems the direction they are going.

I like to think of llms as seamless plagiarism machines.

Re: The unbearable cheapness of open weight models

#176

Post by tomnipotent »

chairmansteve wrote: post #172
dada216 wrote: post #152

Earlier quoted context omitted.

I have deployed very successful LLM based software that reads sales people emails and inserts orders, or stuff akin to orders, in the rest of the systems. Can you write me a regex that parses a messy human email thread and produces a clean JSON with all the order details? It's been working for a year with less than 3% error rate, better than what the humans themselves were doing.

Interesting. A web form completed by the sales people would not have sufficed? I guess it's the backend interfaces to multiple systems that's the difficult part?

A web form assumes the input is or can be made structured, or that you have a user that is willing to fill in a form (vs. writing an email). An LLM is great at taking free-form/unstructured content and mapping it nicely to a structured surface area.

Re: The unbearable cheapness of open weight models

#178

Post by tomnipotent »

rcxdude wrote: post #164
therobots927 wrote: post #140

Earlier quoted context omitted.

Are you an anthropic insider or something? Because if you are you should delete this comment. If you aren’t then you don’t know what the hell you’re talking about.

For one point, you can look at the costs of similarly sized open-source models from inference providers (which are only making money on the markup on the compute), and compare with anthropic's prices. There's a pretty big price difference there and it would be hard to believe that anthropic's models are that much more expensive to run than those models.

Those prices don't need to bake in the training cost, since it was eaten by someone else (whomever trained the open source models). Anthropic et al. need to price in the whole lifecycle.

Re: The unbearable cheapness of open weight models

#179

Post by danny_codes »

Tuna-Fish wrote: post #112

> What worries me about this is that Anthropic and OpenAI seem to have backed themselves into a corner of high costs. Can they reasonably decrease their prices by 20-50x to compete with DeepSeek or Xiaomi’s Mimo? They have high prices, not high costs. They will obviously keep prices as high as they can for as long as they can, while keeping demand up. Once demand starts to fall, so will the prices. > Are these models…

So they reduce their prices 50x and their revenue drops to $1B a year? And they're supposed to be worth $1T each? If there is an open model offering at the cost of inference + whatever margin you need to stay alive, then as inference falls Anthropic/OpenAI revenue also falls.

What's the end-game? It sounds like in this business, if you are right, your revenues drop every year.

Post reply on HN