Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why current LLM costs are not sustainable

aditya.patadia.org

Post reply on HN
Print view View original

91–100 of 216 posts

Re: Why current LLM costs are not sustainable

#91

Post by player1234 »

ReptileMan wrote: post #10
KronisLV wrote: post #6

> To give an example, just doing Typescript type fixes with this model across 50 files cost me $54 this afternoon. If you can use a subscription with any of the SOTA models, do that. Instead of around 4k EUR in token costs, my Opus usage costs me 108 EUR (with taxes) per month with their Max 5x plan. It's the same with OpenAI, those are heavily subsidized. It doesn't make sense to pay per-token, unless you must. > Wh…

Why do you think that subscriptions are subsidized and not that enterprise tokens are sold at 3000% margin? There are few enough frontier labs that cartel is possible.

[dead]

Re: Why current LLM costs are not sustainable

#92

Post by chilmers »

It's weird to see people claiming that model capabilities are plateauing. It wasn't until late last year that we even had strong coding models. Imagine if, less than a year after the first iPhone launched, people claimed that smartphone capabilities were "plateauing" because Apple hadn't yet launched a new phone. And it seems the issue is less than "models aren't getting better" than, "models are good enough to handle 99% of the coding tasks people give to them".

Re: Why current LLM costs are not sustainable

#93

Post by winternewt »

erfgh wrote: post #80
LUmBULtERA wrote: post #74

Earlier quoted context omitted.

>3. We're massively overusing SOTA models. As long as you're on a subsidized subscription, you can use Claude Opus 4.8 high to write blog article meta descriptions. If you paid by token, you wouldn't do that. This idea that the subscriptions are subsidized is repeated over and over, but I've never seen any proof of this. It seems to be entirely based on the inferred API cost the subscription usage could give you, but…

They are subsidized by the huge losses incurred by the AI companies.

From the article:

> What is happening here is that leading AI labs are charging not only for inference but also for research in model architecture, training data collection and curation, model training cost (which can be tens or even hundreds of millions of dollars), paying their employees and recovering the marketing costs.

That's what's being subsidized.

Re: Why current LLM costs are not sustainable

#94

Post by arbayi »

byzantinegene wrote: post #72
arbayi wrote: post #33

i think we have the causation backwards here. llms aren't expensive because they have to be — they're expensive because we keep reaching for the expensive model instead of putting any effort into making the cheap one good enough. a surprisingly large fraction of production workloads can be handled by smaller models with the right scaffolding. it's often easier to switch to a larger model than to engineer those pieces…

Unfortunately, the economics of what you suggest do not justify the trillion dollar valuations of OpenAI and Anthropic.

[flagged]

Re: Why current LLM costs are not sustainable

#95

Post by oezi »

yturijea wrote: post #68

I am using perhaps 15% of usage count on Claude with just the normal subscription. And I do full time software engineering and would say I use quite a lot of AI input on thoughts, designs and code drafts. So how these companies and people manage to use these absurd amount of tokens is a mystery to me. It feels like this are just running huge amount of non-vetted data to the LLM's and or running loops against the LLM'…

If you don't reset sessions eagerly or compact regularly it is easy to consume billions in input tokens while Claude churns away.

Re: Why current LLM costs are not sustainable

#96

Post by Aldipower »

KronisLV wrote: post #6

> To give an example, just doing Typescript type fixes with this model across 50 files cost me $54 this afternoon. If you can use a subscription with any of the SOTA models, do that. Instead of around 4k EUR in token costs, my Opus usage costs me 108 EUR (with taxes) per month with their Max 5x plan. It's the same with OpenAI, those are heavily subsidized. It doesn't make sense to pay per-token, unless you must. > Wh…

> If you can use a subscription with any of the SOTA models, do that.

Those subscriptions plans are for private use only! If you are running a business you are not allowed to use them actually. Anyway..

Re: Why current LLM costs are not sustainable

#97

Post by adrian_b »

juleiie wrote: post #9
arjunchint wrote: post #8

There is a wave of users switching over to DeepSeek Flash. There are Reddit threads of users sharing billion token spend for $20. If all of global spend on Anthropic/OpenAI/Gemini APIs just switches over to DeepSeek then easily we can decrease total AI spend by 10x

I am not sure if that is wise. It’s a hostile superpower after all

By "hostile superpower" you mean USA?

As an European, today I certainly classify USA as a "hostile superpower" because the actions from the last few years of both the US government and of certain big US companies have stolen a lot of money directly from my own pocket, by artificially limiting competition in several important markets, like smartphones, SSDs and memory modules, thus greatly raising the prices in comparison with what they would have been in normal market conditions (i.e. if the US government had behaved after the same rules that they had forced upon the other countries for decades, by various methods of propaganda, bribing and blackmailing).

Re: Why current LLM costs are not sustainable

#98

Post by PunchyHamster »

eru wrote: post #37
FinnLobsien wrote: post #35

The problem space has a few aspects: 1. We're still in the "$5 airport Uber" era of LLMs. They're heavily subsidized, and everyone still complains about costs. 2. There hasn't been a real incentive to work on cost optimization for data centers and the hardware they contain. When/if price hikes happen and send people scrambling to use other models or drastically reduce AI usage, this will suddenly need to happen. 3. W…

Mostly agreed, however I'm not sure about 3: I suspect it works like gym memberships, and the companies mostly make their money from people who don't use the subscriptions all that much.

Technically yes but it's not hard to get to $20 plan caps. Till current hardware prices cool down I don't see it being easy to make money on frontier models.

Re: Why current LLM costs are not sustainable

#99

Post by Aldipower »

arjunchint wrote: post #8

There is a wave of users switching over to DeepSeek Flash. There are Reddit threads of users sharing billion token spend for $20. If all of global spend on Anthropic/OpenAI/Gemini APIs just switches over to DeepSeek then easily we can decrease total AI spend by 10x

DS is restricting the "expert" model usage already, because they do not have enough compute.

Re: Why current LLM costs are not sustainable

#100

Post by Hamuko »

chilmers wrote: post #85
erfgh wrote: post #80

Earlier quoted context omitted.

They are subsidized by the huge losses incurred by the AI companies.

Only if those losses are coming from subscriptions, instead of capex and training, which is not at all clear.

I don't understand this argument. How does it make the subscription any less subsidised if the losses are only because developing the product is just so darn expensive?

Feels like arguing that it's not clear if Bugatti's losses came from selling the Veyron instead of designing and developing the Veyron.

Post reply on HN