Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why current LLM costs are not sustainable

aditya.patadia.org

Post reply on HN
Print view View original

201–210 of 216 posts

Re: Why current LLM costs are not sustainable

#201

Post by rightbyte »

usef- wrote: post #168
Yizahi wrote: post #156

Earlier quoted context omitted.

Anyone can claim they are profitable, simply by reclassifying their expenses as some other thing or shuffling them to separate corporate structure. Until we will real financial audit, the CEOs claims are just a hot air.

OpenAI's leaked documents also said OpenAI was profitable on inference. The small resellers of open models have nowhere near the resources to optimise their models or inference and yet usually have a lower cost, why wouldn't the big labs?

The leak really doesn't say that. We have no way of knowing what posts are in the different categories.

Re: Why current LLM costs are not sustainable

#202

Post by piva00 »

hparadiz wrote: post #193
piva00 wrote: post #192

Earlier quoted context omitted.

What? LLMs were designed for text, it's in their name "large language model". Only with specialised encoders like vision transformers they were able to process images as well but you're absolutely wrong about the original design intent. In the end you just added misinformation, just save the comment to your favourites and set a reminder to check it again in a few years like you wanted.

The first technological breakthroughs were with face and red eye detection in 2003. Then object detection between 2008-2012. Text models didn't become useful until about 2016. Please watch the first course of Dr Fei Fei Li's lectures on the subject.

If we want to keep tracing the lineage of AI we'll have to go all the way back to Markov chains from the 70s.

You said LLMs were designed for images which is absolutely incorrect.

Re: Why current LLM costs are not sustainable

#203

Post by raffael_de »

yturijea wrote: post #68

I am using perhaps 15% of usage count on Claude with just the normal subscription. And I do full time software engineering and would say I use quite a lot of AI input on thoughts, designs and code drafts. So how these companies and people manage to use these absurd amount of tokens is a mystery to me. It feels like this are just running huge amount of non-vetted data to the LLM's and or running loops against the LLM'…

coding harnesses loading entire code bases for every task - at least that's my theory because I also never even get close to the limits of my 20-sth bucks level subscriptions.

Re: Why current LLM costs are not sustainable

#204

Post by xboxnolifes »

heyoni wrote: post #101
LUmBULtERA wrote: post #74

Earlier quoted context omitted.

>3. We're massively overusing SOTA models. As long as you're on a subsidized subscription, you can use Claude Opus 4.8 high to write blog article meta descriptions. If you paid by token, you wouldn't do that. This idea that the subscriptions are subsidized is repeated over and over, but I've never seen any proof of this. It seems to be entirely based on the inferred API cost the subscription usage could give you, but…

What assumptions are needed for inferring cost based on api pricing?

API prices are consumer prices, not costs.

Re: Why current LLM costs are not sustainable

#205

Post by JBlue42 »

LUmBULtERA wrote: post #74
FinnLobsien wrote: post #35

The problem space has a few aspects: 1. We're still in the "$5 airport Uber" era of LLMs. They're heavily subsidized, and everyone still complains about costs. 2. There hasn't been a real incentive to work on cost optimization for data centers and the hardware they contain. When/if price hikes happen and send people scrambling to use other models or drastically reduce AI usage, this will suddenly need to happen. 3. W…

>3. We're massively overusing SOTA models. As long as you're on a subsidized subscription, you can use Claude Opus 4.8 high to write blog article meta descriptions. If you paid by token, you wouldn't do that. This idea that the subscriptions are subsidized is repeated over and over, but I've never seen any proof of this. It seems to be entirely based on the inferred API cost the subscription usage could give you, but…

Because no one knows what the true cost is, especially with how all the money is circular.

For example, Microsoft 365 Copilot. A company might get a significant discount on the price per user for that SKU. What does that translate to? OpenAI is the actual brain behind it. So who ends up getting the money at the end?

https://blogs.microsoft.com/blog/2026/04/27/the-next-phase-o...

OpenAI needs Microsoft's infra for training and market penetration. MS needs something AI to slap onto their products until they develop their own in-house.

Now, whether it's actually necessary or not for an enterprise is a question. There's a lot of FOMO spending. Maybe 5-10% of your workforce are programmers that need great AI. The rest of the admin, finance, other people? TBD.

Re: Why current LLM costs are not sustainable

#206

Post by arjunchint »

szszrk wrote: post #164
arjunchint wrote: post #8

There is a wave of users switching over to DeepSeek Flash. There are Reddit threads of users sharing billion token spend for $20. If all of global spend on Anthropic/OpenAI/Gemini APIs just switches over to DeepSeek then easily we can decrease total AI spend by 10x

v4 flash was really, really good in practice for me. While on openrouter it's around 1/100th of what the "SOTA" models cost. But billions? A bit exaggerated.

Plenty of screenshots in this thread showing usage.

Its the ridiculous cached input token price of $0.00028/1 M

https://www.reddit.com/r/DeepSeek/comments/1twesxe/comment/o...

Re: Why current LLM costs are not sustainable

#207

Post by fragmede »

g-b-r wrote: post #183
fragmede wrote: post #137

Earlier quoted context omitted.

The equivalent is when Amazon was running a loss because they were spending all their money on building warehouses. It exactly make sense, but that's the argument.

How often does Amazon have to rebuild their warehouses?

> It (doesn't) exactly make sense

I missed the word don't, originally. Apologies.

Re: Why current LLM costs are not sustainable

#208

Post by sophrosyne42 »

adrian_b wrote: post #110
sophrosyne42 wrote: post #34

Earlier quoted context omitted.

We are hostile to each other. It's ignorant or propagandistic to pretend it is only one sided. The concern is valid, if vague and unproven

USA is hostile to the entire world, because the US actions already for several years, but especially during the last year, have caused global price rises in more and more product categories, starting with smartphones, then with SSDs, then with DRAM and HDDs, and eventually with almost everything that is affected by energy costs. This is not some hypothetical hostility, but billions of humans from all over the world h…

China regularly conducts military, even live fire, exercises in the waters the South China Sea and surrounding waters, fishes and uses these water illegally, it helps Russia with its war against Ukraine, has provided assistance to Iran and continues to do so, has set up illegal policing units inside of other countries, conducts regular espionage, influence campaigns, propaganda, and infiltrates the political structures of other countries, and more. Unlike much (though not all) of what you blame the US for, these are all direct decisions by the state China rather than developments in its respective market by private actors.

China's hostility is not hypothetical. The US's hostility is not universal. I think my point about ignorance or propaganda is proven by your statement.

Re: Why current LLM costs are not sustainable

#209

Post by tasuki »

edb_123 wrote: post #147
nDRDY wrote: post #114

Earlier quoted context omitted.

To draw a parallel - airport Ubers are still $5, but you can't buy a 2nd hand prius any more!

Following your parallel: Except the fact that you still need a car in your life, even if you take an uber to the airport when needed :) And in this analogy you need to spend a lot more when buying a car, no matter if it's a new or 2nd hand one, following the price inflation caused by cheap Ubers. So in essence, my question is how much have those cheap Uber rides then cost you in reality, when factoring in the directl…

> Except the fact that you still need a car in your life

No.

Re: Why current LLM costs are not sustainable

#210

Post by sfifs »

FinnLobsien wrote: post #35

The problem space has a few aspects: 1. We're still in the "$5 airport Uber" era of LLMs. They're heavily subsidized, and everyone still complains about costs. 2. There hasn't been a real incentive to work on cost optimization for data centers and the hardware they contain. When/if price hikes happen and send people scrambling to use other models or drastically reduce AI usage, this will suddenly need to happen. 3. W…

> We're still in the "$5 airport Uber" era of LLMs. They're heavily subsidized, and everyone still complains about costs.

This is nonsense that AI providers want to peddle. Inference is wildly gross margin profitable - likely 90%+ gross margins. It's very easy to work out the cost structures bottoms up. All providers can drop costs to a third and still keep positive gross margins.

The problems are 1. It possibly still doesn't pay out on training investment in a reasonable time frame without a massive expansion of the 90% gross margin.

2. There is no moat. As we see Mac Mini & High End GPUs stock outs and the pricing offered by DeepSeek and Qwen, the performance of Open Weight models are good enough that people can and are already shifting many inference workloads out of these 90% margin players

Post reply on HN