Why current LLM costs are not sustainable • Hacker News Board

Re: Why current LLM costs are not sustainable

#111

Post by user43928 » Fri, Jun 26, 2026, 9:52 AM UTC

xienze wrote: post #82

user43928 wrote: post #41

Earlier quoted context omitted.

Opus 4.8 High effort seems adequate for me currently, at API pricing, with a $200/month budget. This is at work where I don't work on greenfield or parallelize feature development. I cannot see the agent burning through $50 for one moderately sized TypeScript cleanup in my setup. This sounds like something that can be improved on OP's side. There have been rumors about a potential Sonnet 5 model release in the near f…

> I cannot see the agent burning through $50 for one moderately sized TypeScript cleanup in my setup. I have absolutely seen stuff like this happen. Think about it, when you point Claude at a bunch of files, it has to suck them all up (tens of thousands of tokens), spend some proportional number of tokens doing stuff, and spit them back out (tens of thousands of tokens) for each pass in the "cleanup" loop. I had a si…

I've spend a week doing just that - I said at API pricing, $200/month currently seems adequate for 2-4 weeks of usage for me at work.

$50 would be 10M input tokens, not tens of thousands.

Re: Why current LLM costs are not sustainable

#112

Post by senectus1 » Fri, Jun 26, 2026, 9:52 AM UTC

I've posted this anecdote before, but i feel its worth posting again because I've seen this spread further.

>amusing side note: >Was in a meeting reviewing a potential new product, it was going well until they showed us that they had added AI to it (of course they have). It was pretty obviously just shoehorned in, and one part of that obviousness was that they had a column that showed how many tokens it took to make each query.

>I asked who is paying for the tokens, they said its included in the license. I said, so is there a budget or is it all you can eat. they said good question they didnt know and would get back to me. I said the reason i asked was just one query there had a 250k token burn on it. and it was a fairly simple query about one device.

>then, one of the execs on their side was heard saying out loud "Why are we even showing this to the customers?"

>it have us quite a chuckle. But lesson learned... the cost of adding AI to anything isnt really being accounted for let alone the true cost of actually running the AI.

>all things AI are going to get more expensive. even if you dont want the AI aspect.

Re: Why current LLM costs are not sustainable

#113

Post by oarsinsync » Fri, Jun 26, 2026, 9:54 AM UTC

LUmBULtERA wrote: post #74

FinnLobsien wrote: post #35

The problem space has a few aspects: 1. We're still in the "$5 airport Uber" era of LLMs. They're heavily subsidized, and everyone still complains about costs. 2. There hasn't been a real incentive to work on cost optimization for data centers and the hardware they contain. When/if price hikes happen and send people scrambling to use other models or drastically reduce AI usage, this will suddenly need to happen. 3. W…

>3. We're massively overusing SOTA models. As long as you're on a subsidized subscription, you can use Claude Opus 4.8 high to write blog article meta descriptions. If you paid by token, you wouldn't do that. This idea that the subscriptions are subsidized is repeated over and over, but I've never seen any proof of this. It seems to be entirely based on the inferred API cost the subscription usage could give you, but…

> This idea that the subscriptions are subsidized is repeated over and over, but I've never seen any proof of this. It seems to be entirely based on the inferred API cost the subscription usage could give you, but there are a lot of assumptions needed for that to follow.

My claude code environment shows me cost per token used in that session, according to API costs. It regularly exceeds $200. I pay $200 a month for my claude subscription. That's fairly obviously subsidised, unless you genuinely believe their unit costs are 100x less than what they're charging.

Re: Why current LLM costs are not sustainable

#114

Post by nDRDY » Fri, Jun 26, 2026, 9:54 AM UTC

edb_123 wrote: post #102

FinnLobsien wrote: post #35

The problem space has a few aspects: 1. We're still in the "$5 airport Uber" era of LLMs. They're heavily subsidized, and everyone still complains about costs. 2. There hasn't been a real incentive to work on cost optimization for data centers and the hardware they contain. When/if price hikes happen and send people scrambling to use other models or drastically reduce AI usage, this will suddenly need to happen. 3. W…

> 1. We're still in the "$5 airport Uber" era of LLMs. They're heavily subsidized, and everyone still complains about costs. How does that figure look if you count in the current unprecedented LLM/AI-driven price inflation on both hardware, services and software? I don't believe we're exactly in the "$5 airport uber" era if you count that into your total.

To draw a parallel - airport Ubers are still $5, but you can't buy a 2nd hand prius any more!

Re: Why current LLM costs are not sustainable

#115

Post by yreg » Fri, Jun 26, 2026, 9:58 AM UTC

winternewt wrote: post #93

erfgh wrote: post #80

Earlier quoted context omitted.

They are subsidized by the huge losses incurred by the AI companies.

From the article: > What is happening here is that leading AI labs are charging not only for inference but also for research in model architecture, training data collection and curation, model training cost (which can be tens or even hundreds of millions of dollars), paying their employees and recovering the marketing costs. That's what's being subsidized.

You are saying it as if those costs were not necessary to provide the service.

Re: Why current LLM costs are not sustainable

#116

Post by LUmBULtERA » Fri, Jun 26, 2026, 9:58 AM UTC

_flux wrote: post #24

ReptileMan wrote: post #10

Earlier quoted context omitted.

Why do you think that subscriptions are subsidized and not that enterprise tokens are sold at 3000% margin? There are few enough frontier labs that cartel is possible.

I think this comes from the idea that serving these tokens without paying for training is already expensive, e.g. https://news.ycombinator.com/item?id=46613887 self-hosted solution might give you only 10-100x more affordable solution at cost . So, given the SOTA providers with even larger models also need to continously be using considerable resources for training their next models, to fund future data centers, and m…

Except there are plenty of inference providers worldwide (including the US) that serve open-weight models that are not subsidized, and are reasonable in cost. Or is your claim that those are all running at a loss?

Re: Why current LLM costs are not sustainable

#117

Post by smokedetector1 » Fri, Jun 26, 2026, 9:58 AM UTC

chilmers wrote: post #85

erfgh wrote: post #80

Earlier quoted context omitted.

They are subsidized by the huge losses incurred by the AI companies.

Only if those losses are coming from subscriptions, instead of capex and training, which is not at all clear.

this argument assumes that capex and training costs will go down over time. but theyll have to keep up with one another and stay on top of latest knowledge so Im not sure if thats true

Re: Why current LLM costs are not sustainable

#118

Post by rimliu » Fri, Jun 26, 2026, 9:58 AM UTC

chilmers wrote: post #92

It's weird to see people claiming that model capabilities are plateauing. It wasn't until late last year that we even had strong coding models. Imagine if, less than a year after the first iPhone launched, people claimed that smartphone capabilities were "plateauing" because Apple hadn't yet launched a new phone. And it seems the issue is less than "models aren't getting better" than, "models are good enough to handl…

People claim what they see. I see no improvement since opus 4.6, quite the opossite.

Re: Why current LLM costs are not sustainable

#119

Post by gck1 » Fri, Jun 26, 2026, 9:59 AM UTC

heyoni wrote: post #101

LUmBULtERA wrote: post #74

Earlier quoted context omitted.

>3. We're massively overusing SOTA models. As long as you're on a subsidized subscription, you can use Claude Opus 4.8 high to write blog article meta descriptions. If you paid by token, you wouldn't do that. This idea that the subscriptions are subsidized is repeated over and over, but I've never seen any proof of this. It seems to be entirely based on the inferred API cost the subscription usage could give you, but…

What assumptions are needed for inferring cost based on api pricing?

That API pricing isn't massively inflated.

Re: Why current LLM costs are not sustainable

#120

Post by xienze » Fri, Jun 26, 2026, 10:00 AM UTC

user43928 wrote: post #111

xienze wrote: post #82

Earlier quoted context omitted.

> I cannot see the agent burning through $50 for one moderately sized TypeScript cleanup in my setup. I have absolutely seen stuff like this happen. Think about it, when you point Claude at a bunch of files, it has to suck them all up (tens of thousands of tokens), spend some proportional number of tokens doing stuff, and spit them back out (tens of thousands of tokens) for each pass in the "cleanup" loop. I had a si…

I've spend a week doing just that - I said at API pricing, $200/month currently seems adequate for 2-4 weeks of usage for me at work. $50 would be 10M input tokens, not tens of thousands.

> I said at API pricing, $200/month

Well I saw $200/month and thought you were talking about a max plan, sorry. But I will say unless you're using that top end model extremely judiciously $200 for 2-4 weeks of work is similarly hard to believe (see the other poster breaking down their usage). What are you typically doing? Must be pretty hardcore stuff if you need to use the baddest available model. How many interactions per day? Care to share your token usage stats?

> $50 would be 10M input tokens, not tens of thousands.

Two things. One, input tokens are but one component, and the cheapest. Output tokens include the tens of thousands being spit out for file changes AND the thinking/crunching that you don't see. And that's the most expensive part. And remember, that's per iteration, not everything is one-shot (especially with tasks like "fix this large part of my codebase).