Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Are OpenAI and Anthropic losing money on inference?

martinalderson.com

Post reply on HN
Print view View original

351–360 of 495 posts

Re: Are OpenAI and Anthropic losing money on inference?

#351

Post by dcre »

Avshalom wrote: post #332
vonneumannstan wrote: post #324

Earlier quoted context omitted.

>Also, in Nike's case, as they grow they get better at making more shoes for cheaper. This is clearly the case for models as well. Training and serving inference for GPT4 level models is probably > 100x cheaper than they used to be. Nike has been making Jordan 1's for 40+ years! OpenAI would be incredibly profitable if they could live off the profit from improved inference efficiency on a GPT4 level model!

>>This is clearly the case ... probably >>OpenAI would be incredibly profitable if they could live off the profit from improved inference efficiency on a GPT4 level model! If gpt4 was basically free money at this point it's real weird that their first instinct was to cut it off after gpt5

I think the idea here is that gpt-5-mini is the cheap gpt-4 quality model they want to serve and make money on.

Re: Are OpenAI and Anthropic losing money on inference?

#352

Post by oblio »

DenisM wrote: post #316
ToucanLoucan wrote: post #235

Earlier quoted context omitted.

So is OpenAI capable of not making a new model at some point? They've been training the next model continuously as long as they've existed AFAIK. Our software house spends a lot on R&D sure, but we're still incredibly profitable all the same. If OpenAI is in a position where they effectively have to stop iterating the product to be profitable, I wouldn't call that a very good place to be when you're on the verge of h…

There’s still untapped value in deeper integrations. They might hit a jackpot of exponentially increasing value from network effects caused by tight integration with e.g. disjoint business processes. We know that businesses with tight network effects can grow to about 2 trillion in valuation.

How would that look with at least 3 US companies, probably 2 Chinese ones and at least 1 European company developing state of the art LLMs?

Re: Are OpenAI and Anthropic losing money on inference?

#353

Post by dcre »

DenisM wrote: post #306
dcre wrote: post #291

Earlier quoted context omitted.

Good point, very possible that Altman is excluding free tier as a marketing cost even if it loses more than they make on paid customers. On the other hand they may be able to cut free tier costs a lot by having the model router send queries to gpt-5-mini where before they were going to 4o.

Free tier provides a lot of training material. Every time you correct ChatGPT on its mistakes you’re giving them knowledge that’s not in any book or website. Thats a moat, albeit one that is slow to build.

That's interesting, though you have to imagine the data set is very low quality on average and distilling high quality training pairs out of it is very costly.

Re: Are OpenAI and Anthropic losing money on inference?

#354

Post by tptacek »

DoesntMatter22 wrote: post #347
trilogic wrote: post #317

Earlier quoted context omitted.

I have to disagree. The biggest cost is still energy consumption, water and maintenance. Not to mention, to keep up with the rivals in incredibly high tempo (so offering billions like Meta recently). Then the cost of hardware that is equal to Nvidia skyrocketing shares :) No one should dare to talk about profit yet. Now is time to grab the market, invest a lot and work hard, hopping for a future profit. The equation…

Is that not baked into the h100 rental costs?

It is.

Re: Are OpenAI and Anthropic losing money on inference?

#355

Post by oblio »

citizenpaul wrote: post #304
aeternum wrote: post #224

Earlier quoted context omitted.

This can be technically true without being actually true. IE OpenAI invests in Cursor/Windsurf/Startups that give away credits to users and make heavy use of inference API. Money flows back to OpenAI then OpenAI sends it back to those companies via credits/investment $. It's even more circular in this case because nvidia is also funding companies that generate significant inference. It'll be quite difficult to figure…

There a journalist ed zittron https://www.wheresyoured.at/ That is an openai skeptic. His research if correct says not only is openai unprofitable but it likely never will be. Can't be ,its various finance ratios make early uber, amazon ect look downright fiscally frugal. He is not a tech person for what that means to you.

Amazon was very frugal. If you look at Amazon losses for the first 10 years, they were all basically under 5% of revenue and many years were break even or slightly net positive.

Uber burnt through a lot of money and even now I'm not sure their lifetime revenue is positive (it's possible that since their foundation they've lost more money than they've made).

Re: Are OpenAI and Anthropic losing money on inference?

#356

Post by dcre »

kgwgk wrote: post #339
dcre wrote: post #219

Earlier quoted context omitted.

ICYMI, Amodei said the same in much greater detail: "If you consider each model to be a company, the model that was trained in 2023 was profitable. You paid $100 million, and then it made $200 million of revenue. There's some cost to inference with the model, but let's just assume, in this cartoonish cartoon example, that even if you add those two up, you're kind of in a good state. So, if every model was a company,…

>> If we didn't pay for training, we'd be a very profitable company. > ICYMI, Amodei said the same No. He says that even paying for training a model is profitable. It makes more revenue that it costs - all things considered. A much stronger claim.

I take them to be saying the same thing — the difference is that Altman is referring to the training of the next model happening now, while Amodei is referring to the training months ago of the model you're currently earning money back on through inference.

Re: Are OpenAI and Anthropic losing money on inference?

#357

Post by ugh123 »

BlindEyeHalo wrote: post #303
_sword wrote: post #276

I've done the modeling on this a few times and I always get to a place where inference can run at 50%+ gross margins, depending mostly on GPU depreciation and how good the host is at optimizing utilization. The challenge for the margins is whether or not you consider model training costs as part of the calculation. If model training isn't capitalized + amortized, margins are great. If they are amortized and need to b…

Why wouldn't you factor in training? It is not like you can train once and then have the model run for years. You need to constantly improve to keep up with the competition. The lifespan of a model is just a few months at this point.

It's possible they factor in training purely as an "R&D" cost and then can tax that development at a lower rate.

Re: Are OpenAI and Anthropic losing money on inference?

#358

Post by citizenpaul »

oblio wrote: post #355
citizenpaul wrote: post #304

Earlier quoted context omitted.

There a journalist ed zittron https://www.wheresyoured.at/ That is an openai skeptic. His research if correct says not only is openai unprofitable but it likely never will be. Can't be ,its various finance ratios make early uber, amazon ect look downright fiscally frugal. He is not a tech person for what that means to you.

Amazon was very frugal. If you look at Amazon losses for the first 10 years, they were all basically under 5% of revenue and many years were break even or slightly net positive. Uber burnt through a lot of money and even now I'm not sure their lifetime revenue is positive (it's possible that since their foundation they've lost more money than they've made).

Exactly Zittrons point.

Re: Are OpenAI and Anthropic losing money on inference?

#359

Post by drob518 »

oblio wrote: post #352
DenisM wrote: post #316

Earlier quoted context omitted.

There’s still untapped value in deeper integrations. They might hit a jackpot of exponentially increasing value from network effects caused by tight integration with e.g. disjoint business processes. We know that businesses with tight network effects can grow to about 2 trillion in valuation.

How would that look with at least 3 US companies, probably 2 Chinese ones and at least 1 European company developing state of the art LLMs?

Like a very over-served market, I think. I see perhaps three survivors long term, or at most one gorilla, two chimps, and perhaps a few very small niche-focused monkeys.

Re: Are OpenAI and Anthropic losing money on inference?

#360

Post by trilogic »

wtallis wrote: post #318
trilogic wrote: post #317

Earlier quoted context omitted.

I have to disagree. The biggest cost is still energy consumption, water and maintenance. Not to mention, to keep up with the rivals in incredibly high tempo (so offering billions like Meta recently). Then the cost of hardware that is equal to Nvidia skyrocketing shares :) No one should dare to talk about profit yet. Now is time to grab the market, invest a lot and work hard, hopping for a future profit. The equation…

> The biggest cost is still energy consumption, water and maintenance. Are you saying that the operating costs for inference exceed the costs of training?

The global cost of inference in both Openai and Anthropic it exceed training cost for sure. The reason is simple: the inference cost grows with requests not with datasets. My math simplified by AI says: Suppose training GPT-like model costs

= $ 10,000,000 C T

=$10,000,000.

Each query costs

= $ 0.002 C I

=$0.002.

Break-even:

> 10,000,000 0.002 = 5,000,000,000

inferences N> 0.002 10,000,000

=5,000,000,000inferences

So after 5 billion queries, inference costs surpass the training cost.

Openai claims it has 100 million users x queries = I let you judge.

Post reply on HN