Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why current LLM costs are not sustainable

aditya.patadia.org

Post reply on HN
Print view View original

141–150 of 216 posts

Re: Why current LLM costs are not sustainable

#141

Post by LUmBULtERA »

_flux wrote: post #125
LUmBULtERA wrote: post #116

Earlier quoted context omitted.

Except there are plenty of inference providers worldwide (including the US) that serve open-weight models that are not subsidized, and are reasonable in cost. Or is your claim that those are all running at a loss?

So they do not train models, and in addition their models are expected to be smaller than SOTA models, although we cannot know for sure by how much. So what's the price difference, 3000x?

My comment is about your statement "serving these tokens without paying for training is already expensive"...

One thing we do know from OpenAI's leaked financial document is that they are already profitable on inference, though that data is not broken down by cost and revenue of API vs. subscription. One important factor is that subscription inference can be optimized in ways to reduce cost (e.g., usage limits, batch optimization around API-prioritized inference, etc...). I think simply we do not know the actual cost of subscription interference for SOTA models.

Re: Why current LLM costs are not sustainable

#142

Post by IsTom »

pydry wrote: post #15
arjunchint wrote: post #8

There is a wave of users switching over to DeepSeek Flash. There are Reddit threads of users sharing billion token spend for $20. If all of global spend on Anthropic/OpenAI/Gemini APIs just switches over to DeepSeek then easily we can decrease total AI spend by 10x

Probably won't be too long before the government decides to block deepseek's website based on "security" concerns.

And then USA will have a disadvantage compared to rest of the world with cheaper LLMs and american AI companies will have tougher time surviving on domestic spend alone.

Re: Why current LLM costs are not sustainable

#143

Post by usef- »

xienze wrote: post #105
eru wrote: post #37

Earlier quoted context omitted.

Mostly agreed, however I'm not sure about 3: I suspect it works like gym memberships, and the companies mostly make their money from people who don't use the subscriptions all that much.

> I suspect it works like gym memberships, and the companies mostly make their money from people who don't use the subscriptions all that much. I think it's like that, but not quite. The people who have a subscription but barely use it were probably never doing any serious work with AI in the first place. I.e., why would they get a subscription when their one or two chat questions (or, "make a picture of me as a supe…

Based on the people I know, they're paying because when ask they want the smartest model to be the one answering. There's still quite a difference between models.

Re: Why current LLM costs are not sustainable

#144

Post by fragmede »

FinnLobsien wrote: post #35

The problem space has a few aspects: 1. We're still in the "$5 airport Uber" era of LLMs. They're heavily subsidized, and everyone still complains about costs. 2. There hasn't been a real incentive to work on cost optimization for data centers and the hardware they contain. When/if price hikes happen and send people scrambling to use other models or drastically reduce AI usage, this will suddenly need to happen. 3. W…

> 1. We're still in the "$5 airport Uber" era of LLMs. They're heavily subsidized, and everyone still complains about costs.

Do they? It's free right now at chat.com. After that it's $20/month which isn't much in the US. Three Starbucks or two meals at McDonald's will run you more than that these days.

Re: Why current LLM costs are not sustainable

#145

Post by LUmBULtERA »

heyoni wrote: post #101
LUmBULtERA wrote: post #74

Earlier quoted context omitted.

>3. We're massively overusing SOTA models. As long as you're on a subsidized subscription, you can use Claude Opus 4.8 high to write blog article meta descriptions. If you paid by token, you wouldn't do that. This idea that the subscriptions are subsidized is repeated over and over, but I've never seen any proof of this. It seems to be entirely based on the inferred API cost the subscription usage could give you, but…

What assumptions are needed for inferring cost based on api pricing?

The API inference cost to customers is not the actual cost of providing inference, and the cost of providing API inference need not be the cost of providing subscriber inference.

Re: Why current LLM costs are not sustainable

#146

Post by LUmBULtERA »

oarsinsync wrote: post #113
LUmBULtERA wrote: post #74

Earlier quoted context omitted.

>3. We're massively overusing SOTA models. As long as you're on a subsidized subscription, you can use Claude Opus 4.8 high to write blog article meta descriptions. If you paid by token, you wouldn't do that. This idea that the subscriptions are subsidized is repeated over and over, but I've never seen any proof of this. It seems to be entirely based on the inferred API cost the subscription usage could give you, but…

> This idea that the subscriptions are subsidized is repeated over and over, but I've never seen any proof of this. It seems to be entirely based on the inferred API cost the subscription usage could give you, but there are a lot of assumptions needed for that to follow. My claude code environment shows me cost per token used in that session, according to API costs. It regularly exceeds $200. I pay $200 a month for m…

The API inference cost to customers is not the actual cost of providing inference, and the cost of providing API inference need not be the cost of providing subscriber inference.

Re: Why current LLM costs are not sustainable

#147

Post by edb_123 »

nDRDY wrote: post #114
edb_123 wrote: post #102

Earlier quoted context omitted.

> 1. We're still in the "$5 airport Uber" era of LLMs. They're heavily subsidized, and everyone still complains about costs. How does that figure look if you count in the current unprecedented LLM/AI-driven price inflation on both hardware, services and software? I don't believe we're exactly in the "$5 airport uber" era if you count that into your total.

To draw a parallel - airport Ubers are still $5, but you can't buy a 2nd hand prius any more!

Following your parallel: Except the fact that you still need a car in your life, even if you take an uber to the airport when needed :)

And in this analogy you need to spend a lot more when buying a car, no matter if it's a new or 2nd hand one, following the price inflation caused by cheap Ubers. So in essence, my question is how much have those cheap Uber rides then cost you in reality, when factoring in the directly related price increases for the things you need and buy? Is it a net positive or negative at the end of the day for anyone other than the very few at the very top of the system?

Re: Why current LLM costs are not sustainable

#148

Post by LUmBULtERA »

erfgh wrote: post #80
LUmBULtERA wrote: post #74

Earlier quoted context omitted.

>3. We're massively overusing SOTA models. As long as you're on a subsidized subscription, you can use Claude Opus 4.8 high to write blog article meta descriptions. If you paid by token, you wouldn't do that. This idea that the subscriptions are subsidized is repeated over and over, but I've never seen any proof of this. It seems to be entirely based on the inferred API cost the subscription usage could give you, but…

They are subsidized by the huge losses incurred by the AI companies.

Data from OpenAI shows their 2025 inference revenue exceeded their cost of inference by a good margin (https://cdn.arstechnica.net/wp-content/uploads/2026/06/opena...). Saying this is being subsidized is like saying any investment in future productive assets is "subsidized".

Re: Why current LLM costs are not sustainable

#149

Post by hparadiz »

t-writescode wrote: post #131
hparadiz wrote: post #123

Earlier quoted context omitted.

That's what they want to charge you. Not the actual cost. The actual cost is a gpu that's probably already paid off and about $2 of electricity

Most prices, like GPUs, are amortized over several years, when doing the calculus. Maybe they're already paid off, maybe they aren't. I would lean toward "aren't".

[dead]

Re: Why current LLM costs are not sustainable

#150

Post by LUmBULtERA »

yreg wrote: post #115
winternewt wrote: post #93

Earlier quoted context omitted.

From the article: > What is happening here is that leading AI labs are charging not only for inference but also for research in model architecture, training data collection and curation, model training cost (which can be tens or even hundreds of millions of dollars), paying their employees and recovering the marketing costs. That's what's being subsidized.

You are saying it as if those costs were not necessary to provide the service.

OpenAI inference revenue exceeds its cost of inference by a good margin in 2025 (https://cdn.arstechnica.net/wp-content/uploads/2026/06/opena...)
Post reply on HN