Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why current LLM costs are not sustainable

aditya.patadia.org

Post reply on HN
Print view View original

151–160 of 216 posts

Re: Why current LLM costs are not sustainable

#151

Post by hparadiz »

FinnLobsien wrote: post #47
eru wrote: post #37

Earlier quoted context omitted.

Mostly agreed, however I'm not sure about 3: I suspect it works like gym memberships, and the companies mostly make their money from people who don't use the subscriptions all that much.

I think the problem is that the companies mostly don't make money, period. They may have better unit economics on underused subscriptions, but I don't see a world in which OAI/Anthropic don't heavily tighten the screws in the future. Right now it's silly to default to frontier models, but it won't bankrupt your company. I believe in the short-medium term future, we'll need to be more deliberate about model choices. I…

[flagged]

Re: Why current LLM costs are not sustainable

#152

Post by offby_one »

This entire discussion is on the supply side chips, available weight, and switching cost. What is missing is the demand aspect. The firms that will be the most impacted when there is a drop in price are not Uber or Microsoft; it is that small privately-held firm using spreadsheets and hunches to make business decisions and cannot afford to invest in AI due to the high costs involved currently. The minute inference becomes affordable, an entirely new set of users joins the scene. The real issue is not what will happen to the laboratories it's what gets built for that next wave and whether it's actually useful or just cheaper versions of what already exists.

Re: Why current LLM costs are not sustainable

#154

Post by jijijijij »

winternewt wrote: post #93
erfgh wrote: post #80

Earlier quoted context omitted.

They are subsidized by the huge losses incurred by the AI companies.

From the article: > What is happening here is that leading AI labs are charging not only for inference but also for research in model architecture, training data collection and curation, model training cost (which can be tens or even hundreds of millions of dollars), paying their employees and recovering the marketing costs. That's what's being subsidized.

Is this supposed to be some sort of gotcha? Apart from research and marketing, that's operational costs. I mean, every product could be cheaper, if you didn't have to pay for employees and means of production.

Re: Why current LLM costs are not sustainable

#155

Post by usef- »

rimliu wrote: post #118
chilmers wrote: post #92

It's weird to see people claiming that model capabilities are plateauing. It wasn't until late last year that we even had strong coding models. Imagine if, less than a year after the first iPhone launched, people claimed that smartphone capabilities were "plateauing" because Apple hadn't yet launched a new phone. And it seems the issue is less than "models aren't getting better" than, "models are good enough to handl…

People claim what they see. I see no improvement since opus 4.6, quite the opossite.

That's only a few months old. Just because there's time between big releases doesn't mean progress stopped.

Fable seemed very clearly a step up in my one afternoon of usage. I gave it several bugs that other models had failed at repeatedly (in a mess of a vibe coded side project) and it fixed them each in one prompt.

Re: Why current LLM costs are not sustainable

#156

Post by Yizahi »

usef- wrote: post #135
erfgh wrote: post #80

Earlier quoted context omitted.

They are subsidized by the huge losses incurred by the AI companies.

Anthropic have claimed they expect their first profitable quarter this year. As far as we can infer their current API prices have decent margins.

Anyone can claim they are profitable, simply by reclassifying their expenses as some other thing or shuffling them to separate corporate structure. Until we will real financial audit, the CEOs claims are just a hot air.

Re: Why current LLM costs are not sustainable

#157

Post by jijijijij »

LUmBULtERA wrote: post #150
yreg wrote: post #115

Earlier quoted context omitted.

You are saying it as if those costs were not necessary to provide the service.

OpenAI inference revenue exceeds its cost of inference by a good margin in 2025 ( https://cdn.arstechnica.net/wp-content/uploads/2026/06/opena... )

Great, but that's only a part of operational costs. A craftsman's revenue may exceed the electricity bill for the power drill, doesn't mean the business is sustainable.

Re: Why current LLM costs are not sustainable

#158

Post by Yizahi »

chilmers wrote: post #85
erfgh wrote: post #80

Earlier quoted context omitted.

They are subsidized by the huge losses incurred by the AI companies.

Only if those losses are coming from subscriptions, instead of capex and training, which is not at all clear.

"Our 2015 car models are totally profitable if we will just stop making new cars and continue producing only 2015 models for the next decade."

Re: Why current LLM costs are not sustainable

#159

Post by KronisLV »

user43928 wrote: post #140
KronisLV wrote: post #104

Earlier quoted context omitted.

> I cannot see the agent burning through $50 for one moderately sized TypeScript cleanup in my setup. Here's my usage, from the ccusage tool (slightly shortened for readability): ┌──────────┬───────────────┬────────────┬─────────────┬─────────────┬───────────────┬────────────────┬────────────────┬─────────────┐ │ Month │ Agent │ Models │ Input │ Output │ Cache Create │ Cache Read │ Total Tokens │ Cost (USD) │ ├──────…

Interestingly it seems 80% of the cost is in the cached tokens. I do not know whether that is typical, or indicative of conversations with too many turns. Not that I would worry about this on a subscription plan, but at work where we are billed at API rates, I try to move to new conversations as often as possible.

For agentic development upwards of 90% is pretty normal!

For example, if you make Claude Code explore a codebase, write a plan based on it and your requirements, do a few iterations of further specifying and altering it, and afterwards let it work for let's say 2-4 hours.

Sub-agents and dynamic workflows do alter the numbers a bit, but not to a crazy degree in the long run.

Re: Why current LLM costs are not sustainable

#160

Post by sharpshadow »

LUmBULtERA wrote: post #124
pydry wrote: post #15

Earlier quoted context omitted.

Probably won't be too long before the government decides to block deepseek's website based on "security" concerns.

Deepseek's models are open-weight and hosted all over the world, how would blocking deepseek's web sight do anything to stop its model's use?

Deepseek will be sanctioned and therefore no provider will offer it anymore. Only way to use it then will be private but even that will be forbidden if it gets classified as threat to national security.
Post reply on HN