Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AI's Affordability Crisis

blog.dshr.org

Post reply on HN
Print view View original

91–100 of 436 posts

Re: AI's Affordability Crisis

#91

Post by CuriouslyC »

steveBK123 wrote: post #54
piker wrote: post #49

Earlier quoted context omitted.

I.e., the demand for programming tokens turns out to be quite elastic.

I would imagine it only gets worse in the face of good-enough open/chinese/local models too right? Microsoft adding Deepseek support already as I recall? That is - for any definition of "they are behind X months" then eventually they get to the point Claude was in January when the world freaked out, but at 1/10th the cost. A lot of firms are going to mandate that is good enough for their developers.

100%. There will be strict quotas on the expensive models and day to day work will be done on the cheap models that are "good enough" with escalation to the metered models when the cheaper options are spinning their wheels. Eventually the US frontier lab APIs will only get the most heavily triaged work that multiple tiers of cheaper Chinese open weight models have failed on.

And of course the C-suite will have unlimited access to Mythos tier models, which they'll use to summarize reports, while passing down mandates to rank and file to increase usage of less expensive models.

Re: AI's Affordability Crisis

#92

Post by evrydayhustling »

The willingness to throw capital at AI is definitely doing some crazy things, but this article has some bad takes on the data.

> [Ratio of per-token cost to subscription cost] means Anthropic is subsidizing their enterprise customers by up to 40 times, and OpenAI up to 70 times

Actually, they could be subsidizing by more (if they are taking a loss on API), or not at all (if they are soaking API customers by a massive margin).

Separately, these subscriptions get sold to large groups with varying usage, so it's crazy to model assuming every subscription is maxed out. Banks, gyms, and many other businesses work this way, offering consumers flexible access to services that they will realistically use in bursts. It's not always worth the complexity to prevent overuse by a small minority. You can feel like this kind of business model isn't as transparent, but it's silly to pretend it can't work.

> OpenAI spent 44% of their revenue [$5.3B] on sales and marketing! The hype needed to keep the AI bubble inflated is incredibly expensive.

Over that same period (2025), OpenAI added $10B in realized revenue and $14B in run-rate. Sounds like they're getting >2X return within 12 months of those go-to-market dollars. Compare that to like, any other business.

> Thus in recent weeks the idea that Generative AI (LLMs for short) is too expensive has been all over mainstream business media.

Would it be smarter for these companies never to test customers' price tolerance? The quotes following this make it seem like the companies are getting important information about the nature of that price tolerance, and preparing to react. This is the work markets do on both sides to understand the value of a new product.

There are lots of good arguments about AI overinflation, but in order for them to be useful, they have to be rigorous and targeted.

Re: AI's Affordability Crisis

#93

Post by minraws »

827a wrote: post #71

> Zitron's numbers don't tell us the real cost of generating tokens but, subject to the assumption that the platforms are not subsidizing the token price, that means Anthropic is subsidizing their enterprise customers by up to 40 times, and OpenAI up to 70 times Neither Anthropic nor OpenAI are subsidizing enterprise customers. Neither Anthropic nor OpenAI allow Business nor Enterprise customers access to the high va…

Given my experience with hosting these models at scale, working and optimizing load, I don't think the margins are nearly as high as 75% if the models are as big as people often claim.

Only reason deepseek is so cheap is because well I don't know, but actual pricing should be around their initial price which was 4x, at that price you have a healthy 25-50% margin based on occupancy, given the deepseek v4 is a very sparse moe model.

GLM 5.2 for example doesn't have more than 30-50% margins that's assuming old pricing for GPUs, current inflated GPU pricing well I am certain the margins must be lower. Ofc you can host for cheaper with quantization, and if you have very consistent capacity/utilization, which is not the norm with AI workloads.

Overall for large models like GPT 5.5 or Opus there must be healthier margins of around 50-70% assuming GPU pricing didn't increase for these companies. Even if it did 30-40% margin should be possible, even in worst case assuming all GPU they had saw a jump in pricing.

For smaller models it's hard to say, I would guess 20% but these models might be much smaller than I suspect, then it might be double that.

Note the issue is less intelligent tokens don't linearly scale down in memory usage, which is the biggest pain point of serving models. Context sizes have fucked us all.

Also anyone claiming OAI makes less margins on APIs or stuff might be wrong given they are on much lower context size, 1M context definitely is a lot more expensive to serve especially with smaller models like sonnet.

Re: AI's Affordability Crisis

#94

Post by jimbokun »

martinald wrote: post #57
jimbokun wrote: post #33

Earlier quoted context omitted.

Then what are the real costs?

Wrote this a while back. https://martinalderson.com/posts/no-it-doesnt-cost-anthropic... OpenRouter is the best guide to real costs.

Thanks, that’s exactly what I was looking for!

And much more informative than the speculation and guessing in the article.

Re: AI's Affordability Crisis

#95

Post by CuriouslyC »

SpicyLemonZest wrote: post #85
rhinoceraptor wrote: post #62

Earlier quoted context omitted.

How are Anthropic and OpenAI going to compete on price when they're both already deeply unprofitable?

They may not be able to! It's pretty widely acknowledged, for example, that if there's some surprising plateau hiding around the corner they're both going to fail. But that could mean that they're over charging for AI usage to get research money and sustainable rates are lower rather than higher.

The whole hidden plateau hypothesis is kinda bunk, because we're already pretty far in a plateau for general knowledge/question answering, but there are many subdomains where we can push model capabilities, and as we saturate one subdomain we can just shift to another economically valuable one.

There isn't one AI intelligence S curve, there are thousands of them, and they're mostly invisible in the major benchmarks, but for someone trying to do work in that specific area of capability, the progress is transformative.

Re: AI's Affordability Crisis

#96

Post by rich_sasha »

knuckleheads wrote: post #11

Shouldn't we know a better answer to these questions once Anthropic's IPO materials surface publicly? I understand, and maybe even expect, SpaceX's materials to be all over the place and skate on by any discussion of unit economics, but the nerds over at Anthropic might just be forthright enough to just tell us what their margin is on tokens as part of their IPO.

To be honest, making sense of finances of fully public companies is often hard, because in practice, accounting is hard. How you account for depreciacion, cost, investment, fixed vs marginal costs is in practice fluid, companies have an incentive to make it look attractive, while also optimising for tax and shifting revenue around to narrowly beat analyst recommendations.

Here's a concrete example. Does some random AI company make operating profit on inference? I.e. if you only kept marginal costs, would you make a profit?

Well, depends what you account as your costs. If you're using hand-me-down hardware from previous generation's training, how much do you charge yourself internally for it? Maybe you show less, so investors take solace in profitable inference, even if you're losing money overall. How exactly are you accounting for electricity costs between training and inference? Is your army of SREs mostly servicing training new models (R&D expenditure) or inference (operating cost)?

This even has a name, and is called the "big bath" approach. If investors expect one part of your business to be a fiscal black hole, just shove all your costs there. They are accepting of it, and you make the rest of the business look better.

I'm not accusing AI companies of cooking the books, rather I'm trying to highlight you could see all the cash flows and still not know how much money is made or lost where.

Re: AI's Affordability Crisis

#97

Post by Quarrelsome »

Is it not also possible that some of the shift is a consequence of increase of use? While we can be extremely cynical at the finances at play, the lock down and increase of token pricing might be demonstrating a burgeoning demand, which would be a positive indicator.

Re: AI's Affordability Crisis

#98

Post by zytoon »

This summarizes half of the entire AI scene as these guys generate content to paint the entire world the way like to: US equity markets are facing three IPOs .. each led by a world-class bullshitter”.

Re: AI's Affordability Crisis

#100

Post by intrasight »

rhinoceraptor wrote: post #62
tacone wrote: post #27

My take is that Anthropic and OpenAI simply are NOT competing on price. 2 big players are often not enough to create tension on price. Chinese models and open model providers are, indeed, competing on price, and the difference shows.

How are Anthropic and OpenAI going to compete on price when they're both already deeply unprofitable?

There is no moat until a company achieves RSI and/or AGI, and the one that does succeed in moat-making will do so by hacking into and destroying their competitor's infrastructure.

Once moat is achieved, you don't have to compete on price. Of course it'll be academic because the AI will probably destroy all of us.

Post reply on HN