Why current LLM costs are not sustainable • Hacker News Board

Re: Why current LLM costs are not sustainable

#161

Post by Paradigma11 » Fri, Jun 26, 2026, 10:56 AM UTC

yreg wrote: post #115

winternewt wrote: post #93

Earlier quoted context omitted.

From the article: > What is happening here is that leading AI labs are charging not only for inference but also for research in model architecture, training data collection and curation, model training cost (which can be tens or even hundreds of millions of dollars), paying their employees and recovering the marketing costs. That's what's being subsidized.

You are saying it as if those costs were not necessary to provide the service.

They are not. They are necessary for the development of future models, which does not influence the availability of the current ones. Plus you have chinese models distilling current SOTA for pennies on the dollar, so as a consumer I never will be worse off in the long (1-2 years) run.

Re: Why current LLM costs are not sustainable

#162

Post by kamaal » Fri, Jun 26, 2026, 10:58 AM UTC

yturijea wrote: post #68

I am using perhaps 15% of usage count on Claude with just the normal subscription. And I do full time software engineering and would say I use quite a lot of AI input on thoughts, designs and code drafts. So how these companies and people manage to use these absurd amount of tokens is a mystery to me. It feels like this are just running huge amount of non-vetted data to the LLM's and or running loops against the LLM'…

>>So how these companies and people manage to use these absurd amount of tokens is a mystery to me.

Absolutely!

I know some colleagues who are routinely spending thousands of dollars worth of tokens, I can't see to even max out the subscription limits even if Im working all the time. Curiously enough their output is lower too.

Re: Why current LLM costs are not sustainable

#163

Post by KronisLV » Fri, Jun 26, 2026, 10:59 AM UTC

ReptileMan wrote: post #10

KronisLV wrote: post #6

> To give an example, just doing Typescript type fixes with this model across 50 files cost me $54 this afternoon. If you can use a subscription with any of the SOTA models, do that. Instead of around 4k EUR in token costs, my Opus usage costs me 108 EUR (with taxes) per month with their Max 5x plan. It's the same with OpenAI, those are heavily subsidized. It doesn't make sense to pay per-token, unless you must. > Wh…

Why do you think that subscriptions are subsidized and not that enterprise tokens are sold at 3000% margin? There are few enough frontier labs that cartel is possible.

The value of tokens isn't necessarily objective and there's a bunch of factors that go into it: infrastructure costs and deprecation, the operational expenses for that and personnel, as well as a bunch of R&D for training new SOTA models that others can just distill at least partially.

The fact that they'll milk corpos that actually have money is obvious, compared to me because I'm broke, as are many other subscription users. The large AI labs don't seem to be profitable so I bet for the regular users there's plenty of subsidizing going on to at least get people to use the tech (and maybe that'd lead to some conversions at work or API usage eventually):

> OpenAI's net loss ballooned from $5 billion in 2024 to a staggering $39 billion last year, as it continued to spend heavily on AI model development and securing compute capacity, the Financial Times reported on Tuesday, citing audited financial figures confirmed by its sources.

https://finance.yahoo.com/markets/stocks/articles/openai-fin...

Inference itself might be profitable, but is just funding the training and other stuff.

Re: Why current LLM costs are not sustainable

#164

Post by szszrk » Fri, Jun 26, 2026, 11:02 AM UTC

arjunchint wrote: post #8

There is a wave of users switching over to DeepSeek Flash. There are Reddit threads of users sharing billion token spend for $20. If all of global spend on Anthropic/OpenAI/Gemini APIs just switches over to DeepSeek then easily we can decrease total AI spend by 10x

v4 flash was really, really good in practice for me. While on openrouter it's around 1/100th of what the "SOTA" models cost.

But billions? A bit exaggerated.

Re: Why current LLM costs are not sustainable

#165

Post by kamaal » Fri, Jun 26, 2026, 11:03 AM UTC

_pdp_ wrote: post #87

yturijea wrote: post #68

I am using perhaps 15% of usage count on Claude with just the normal subscription. And I do full time software engineering and would say I use quite a lot of AI input on thoughts, designs and code drafts. So how these companies and people manage to use these absurd amount of tokens is a mystery to me. It feels like this are just running huge amount of non-vetted data to the LLM's and or running loops against the LLM'…

I mean have you tried to tokenmax? It is not that hard. Just launch 10 different windows and make sure to loop back in after every turn and you will be burning billions of tokens per month in no time.

The question is what work do you do, that burns so many tokens.

Are in you sending it to work in nested for loops? If yes, what sort of work would that be?

Re: Why current LLM costs are not sustainable

#166

Post by jijijijij » Fri, Jun 26, 2026, 11:07 AM UTC

nok22kon wrote: post #129

walrus01 wrote: post #3

I have already seen a number of people doing the math on what it would take for hardware to self host a Q8XL quantization of GLM5.2 shared between N numbers of people. There's additional advantages that everything you query, all of your context cache and everything it outputs stays private and can't be arbitrarily turned off by external interference. Personally I think it would be a fairly good bet that something wit…

the same argument was made 2 years ago: "in 2 years we'll be able to run GPT-4 level models on an expensive laptop, most people will be using this instead of the fancy cloud models". we are there, Gemma4/Qwen3.6 are GPT-4 level models runnable on a fancy laptop. but expectations shifted, nobody wants a GPT-4 level model anymore

Well, then you also have to assume higher expectations put on commercial offers raising operational costs. So, your objection really is a moot point.

Re: Why current LLM costs are not sustainable

#167

Post by kamaal » Fri, Jun 26, 2026, 11:09 AM UTC

sharpshadow wrote: post #160

LUmBULtERA wrote: post #124

Earlier quoted context omitted.

Deepseek's models are open-weight and hosted all over the world, how would blocking deepseek's web sight do anything to stop its model's use?

Deepseek will be sanctioned and therefore no provider will offer it anymore. Only way to use it then will be private but even that will be forbidden if it gets classified as threat to national security.

>>Deepseek will be sanctioned and therefore no provider will offer it anymore.

That is possible inside the US. How do you do it all over the world? You have to convince every country in the world not use frontier models? Even worse how do you convince all the countries to not build their own models?

Re: Why current LLM costs are not sustainable

#168

Post by usef- » Fri, Jun 26, 2026, 11:13 AM UTC

Yizahi wrote: post #156

usef- wrote: post #135

Earlier quoted context omitted.

Anthropic have claimed they expect their first profitable quarter this year. As far as we can infer their current API prices have decent margins.

Anyone can claim they are profitable, simply by reclassifying their expenses as some other thing or shuffling them to separate corporate structure. Until we will real financial audit, the CEOs claims are just a hot air.

OpenAI's leaked documents also said OpenAI was profitable on inference. The small resellers of open models have nowhere near the resources to optimise their models or inference and yet usually have a lower cost, why wouldn't the big labs?

Re: Why current LLM costs are not sustainable

#169

Post by buynlarge » Fri, Jun 26, 2026, 11:13 AM UTC

Regarding the 'no more training data' point.

I agree that text capabilities are maybe hitting the limit of available training data.

But, the big AI platforms are now being used by so many people for so many things that this becomes the new data source for new and different types of abilities.

Theres also the fact that the AI story means they've been able to fund huge data center builds and hardware innovation. This will lift the existing AI/ML applications (e.g. robotics, sensing) in themselves, as well as the fact that they can be integrated with the text models in probably really useful ways.

So I think, maybe text abilities are nearing the end, but intelligence and other interfaces with the real world still have a lot of space to grow.

Re: Why current LLM costs are not sustainable

#170

Post by LUmBULtERA » Fri, Jun 26, 2026, 11:15 AM UTC

jijijijij wrote: post #157

LUmBULtERA wrote: post #150

Earlier quoted context omitted.

OpenAI inference revenue exceeds its cost of inference by a good margin in 2025 ( https://cdn.arstechnica.net/wp-content/uploads/2026/06/opena... )

Great, but that's only a part of operational costs. A craftsman's revenue may exceed the electricity bill for the power drill, doesn't mean the business is sustainable.

Day 2 the craftsman has not made up for the investment/loss of their equipment. Not a useful example.