Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why current LLM costs are not sustainable

aditya.patadia.org

Post reply on HN
Print view View original

101–110 of 216 posts

Re: Why current LLM costs are not sustainable

#101

Post by heyoni »

LUmBULtERA wrote: post #74
FinnLobsien wrote: post #35

The problem space has a few aspects: 1. We're still in the "$5 airport Uber" era of LLMs. They're heavily subsidized, and everyone still complains about costs. 2. There hasn't been a real incentive to work on cost optimization for data centers and the hardware they contain. When/if price hikes happen and send people scrambling to use other models or drastically reduce AI usage, this will suddenly need to happen. 3. W…

>3. We're massively overusing SOTA models. As long as you're on a subsidized subscription, you can use Claude Opus 4.8 high to write blog article meta descriptions. If you paid by token, you wouldn't do that. This idea that the subscriptions are subsidized is repeated over and over, but I've never seen any proof of this. It seems to be entirely based on the inferred API cost the subscription usage could give you, but…

What assumptions are needed for inferring cost based on api pricing?

Re: Why current LLM costs are not sustainable

#102

Post by edb_123 »

FinnLobsien wrote: post #35

The problem space has a few aspects: 1. We're still in the "$5 airport Uber" era of LLMs. They're heavily subsidized, and everyone still complains about costs. 2. There hasn't been a real incentive to work on cost optimization for data centers and the hardware they contain. When/if price hikes happen and send people scrambling to use other models or drastically reduce AI usage, this will suddenly need to happen. 3. W…

> 1. We're still in the "$5 airport Uber" era of LLMs. They're heavily subsidized, and everyone still complains about costs.

How does that figure look if you count in the current unprecedented LLM/AI-driven price inflation on both hardware, services and software? I don't believe we're exactly in the "$5 airport uber" era if you count that into your total.

Re: Why current LLM costs are not sustainable

#104

Post by KronisLV »

user43928 wrote: post #41
KronisLV wrote: post #6

> To give an example, just doing Typescript type fixes with this model across 50 files cost me $54 this afternoon. If you can use a subscription with any of the SOTA models, do that. Instead of around 4k EUR in token costs, my Opus usage costs me 108 EUR (with taxes) per month with their Max 5x plan. It's the same with OpenAI, those are heavily subsidized. It doesn't make sense to pay per-token, unless you must. > Wh…

Opus 4.8 High effort seems adequate for me currently, at API pricing, with a $200/month budget. This is at work where I don't work on greenfield or parallelize feature development. I cannot see the agent burning through $50 for one moderately sized TypeScript cleanup in my setup. This sounds like something that can be improved on OP's side. There have been rumors about a potential Sonnet 5 model release in the near f…

> I cannot see the agent burning through $50 for one moderately sized TypeScript cleanup in my setup.

Here's my usage, from the ccusage tool (slightly shortened for readability):

  ┌──────────┬───────────────┬────────────┬─────────────┬─────────────┬───────────────┬────────────────┬────────────────┬─────────────┐
  │ Month    │ Agent         │ Models     │       Input │      Output │  Cache Create │     Cache Read │   Total Tokens │  Cost (USD) │
  ├──────────┼───────────────┼────────────┼─────────────┼─────────────┼───────────────┼────────────────┼────────────────┼─────────────┤
  │ 2026-06  │ - Claude      │ - opus-4-8 │  13,635,792 │  32,562,574 │   177,985,265 │  5,265,814,971 │  5,489,998,602 │    $4665.09 │
  └──────────┴───────────────┴────────────┴─────────────┴─────────────┴───────────────┴────────────────┴────────────────┴─────────────┘
Now obviously that is all with the Max 5x subscription, other agents and models excluded.

So per day that'd be around 155 USD (including weekends), which doesn't seem that far off, as long as the example cleanup takes up around 1/3 of one's daily work (or needs a lot of review/test iterations, or needs to review a lot of the existing code etc.).

Re: Why current LLM costs are not sustainable

#105

Post by xienze »

eru wrote: post #37
FinnLobsien wrote: post #35

The problem space has a few aspects: 1. We're still in the "$5 airport Uber" era of LLMs. They're heavily subsidized, and everyone still complains about costs. 2. There hasn't been a real incentive to work on cost optimization for data centers and the hardware they contain. When/if price hikes happen and send people scrambling to use other models or drastically reduce AI usage, this will suddenly need to happen. 3. W…

Mostly agreed, however I'm not sure about 3: I suspect it works like gym memberships, and the companies mostly make their money from people who don't use the subscriptions all that much.

> I suspect it works like gym memberships, and the companies mostly make their money from people who don't use the subscriptions all that much.

I think it's like that, but not quite. The people who have a subscription but barely use it were probably never doing any serious work with AI in the first place. I.e., why would they get a subscription when their one or two chat questions (or, "make a picture of me as a superhero" prompts) per day can be had for free?

Especially with Claude, I think people who subscribe skew very heavily towards people that can very easily make more than $20 worth of queries in a month. And then there's the not-insignificant number of people who are tokenmaxxing.

It's like the gym membership model except ten percent of members are able to spend 72 hours per day at the gym while the rest spend 8 IMO.

Re: Why current LLM costs are not sustainable

#106

Post by avadodin »

dofm wrote: post #57
iamacyborg wrote: post #56

Earlier quoted context omitted.

I follow a guy called Daniel McCarthy on LinkedIn who writes a lot on CLV and that seems to be his take. Even if theoretically you get way more than you pay with subscriptions, the vast majority of people are not power users. https://danielminhmccarthy.com/

The vast majority of active users of ChatGPT could successfully use a model like Gemma 4 12B with agentic search if x86 hardware didn't make that so difficult. Likely even the E4B, which is really both fun and impressive. That is clearly a big component of Apple's bet, anyway.

I have experimented with it and E4b is perfectly capable of being useful if you provide it with ready–to–use skills.

It's still more like programming than telling a chatbot to go make you GTAVI in JavaScript and make sure the graphics are as good as the original.

Maybe a safer prediction would be that most people will be fine just using hybrid agentic programs that run the models locally(probably with extra spyware). I think this is Apple's bet.

Re: Why current LLM costs are not sustainable

#108

Post by hirako2000 »

edb_123 wrote: post #102
FinnLobsien wrote: post #35

The problem space has a few aspects: 1. We're still in the "$5 airport Uber" era of LLMs. They're heavily subsidized, and everyone still complains about costs. 2. There hasn't been a real incentive to work on cost optimization for data centers and the hardware they contain. When/if price hikes happen and send people scrambling to use other models or drastically reduce AI usage, this will suddenly need to happen. 3. W…

> 1. We're still in the "$5 airport Uber" era of LLMs. They're heavily subsidized, and everyone still complains about costs. How does that figure look if you count in the current unprecedented LLM/AI-driven price inflation on both hardware, services and software? I don't believe we're exactly in the "$5 airport uber" era if you count that into your total.

It's just making a parallel. We may be at the 10cent Uber. But oil and labour costs tend to go up, tokens as used today will probably cost what they cost today or less. But we won't just go to the airport, if we can go to Mars we will ask for it.

Re: Why current LLM costs are not sustainable

#109

Post by rimliu »

edb_123 wrote: post #102
FinnLobsien wrote: post #35

The problem space has a few aspects: 1. We're still in the "$5 airport Uber" era of LLMs. They're heavily subsidized, and everyone still complains about costs. 2. There hasn't been a real incentive to work on cost optimization for data centers and the hardware they contain. When/if price hikes happen and send people scrambling to use other models or drastically reduce AI usage, this will suddenly need to happen. 3. W…

> 1. We're still in the "$5 airport Uber" era of LLMs. They're heavily subsidized, and everyone still complains about costs. How does that figure look if you count in the current unprecedented LLM/AI-driven price inflation on both hardware, services and software? I don't believe we're exactly in the "$5 airport uber" era if you count that into your total.

It about what you pay, not about what it costs.

Re: Why current LLM costs are not sustainable

#110

Post by adrian_b »

sophrosyne42 wrote: post #34
KumaBear wrote: post #18

Earlier quoted context omitted.

Hostile? Us or them? I beg to differ who the hostile ones might be.

We are hostile to each other. It's ignorant or propagandistic to pretend it is only one sided. The concern is valid, if vague and unproven

USA is hostile to the entire world, because the US actions already for several years, but especially during the last year, have caused global price rises in more and more product categories, starting with smartphones, then with SSDs, then with DRAM and HDDs, and eventually with almost everything that is affected by energy costs.

This is not some hypothetical hostility, but billions of humans from all over the world have been losing more and more money in recent years and in recent months, from their own pockets, much of which eventually reaches US companies, like Qualcomm and Micron (or South-Korean companies, who have also benefited from the US policies).

Of course, China is not trustworthy, but until now, unless you are a neighbor like Taiwan, their hostility is only hypothetical and in the future, not real and in the present, like for USA.

Post reply on HN