Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Are OpenAI and Anthropic losing money on inference?

martinalderson.com

Post reply on HN
Print view View original

41–50 of 495 posts

Re: Are OpenAI and Anthropic losing money on inference?

#41

Post by esafak »

Aurornis wrote: post #37
martinald wrote: post #13

Earlier quoted context omitted.

(Author here). Yes I am aware of that and did mention it. However - what I wanted to push back in this article was that claude code was completely unsustainable and therefore a flash in the pan and devs aren't at risk (I know you are not saying this). The models as is are still hugely useful, even if no further training was done.

> The models as is are still hugely useful, even if no further training was done. Exactly. The parent comment has an incorrect understanding of what unit economics means. The cost of training is not a factor in the marginal cost of each inference or each new customer. It’s unfortunate this comment thread is the highest upvoted right now when it’s based on a basic misunderstanding of unit economics.

The marginal cost is not the salient factor when the model has to be frequently retrained at great cost. Even if the marginal cost was driven to zero, would they profit?

Re: Are OpenAI and Anthropic losing money on inference?

#44

Post by uludag »

Another comment mentioned the cost associated with the model. Setting that aside, wouldn't we also need to include all of the systems around the inference? I can imagine significant infrastructure and engineering needs around all of these various services, along with the work needed to keep these systems up and running.

Or are these costs just insignificant compared to inference?

Re: Are OpenAI and Anthropic losing money on inference?

#45

Post by gitremote »

These numbers are off.

> $20/month ChatGPT Pro user: Heavy daily usage but token-limited

ChatGPT Pro is $200/month and Sam Altman already admitted that OpenAI is losing money from Pro subscriptions in January 2025:

"insane thing: we are currently losing money on openai pro subscriptions!

people use it much more than we expected."

- Sam Altman, January 6, 2025

https://xcancel.com/sama/status/1876104315296968813

Re: Are OpenAI and Anthropic losing money on inference?

#46

Post by Aurornis »

JCM9 wrote: post #39

With the heat turning up on AI companies to explain how they will land on a viable business model some of this is starting to look like WeWork’s “Community Adjusted EBITA” arguments of “hey if you ignore where we’re losing money, we’re not losing money!” that they made right before imploding. I think most folks understand that pure inference in a vacuum is likely cash flow positive, but that’s not why folks are askin…

A fast growing venture backed startup doing frontier R&D should be losing money overall.

If they weren’t losing money, they wouldn’t be spending enough on R&D. This isn’t some gotcha. It’s what the investors want right now.

Re: Are OpenAI and Anthropic losing money on inference?

#47

Post by losvedir »

JCM9 wrote: post #4

These articles (of which there are many) all make the same basic accounting mistakes. You have to include all the costs associated with the model, not just inference compute. This article is like saying an apartment complex isn’t “losing money” because the monthly rents cover operating costs but ignoring the cost of the building. Most real estate developments go bust because the developers can’t pay the mortgage paym…

I think this is missing the point that the very interesting article makes.

You're arguing that maybe the big companies won't recoup their investment in the models, or profitably train new ones.

But that's a separate question. Whether a model - which now exists! - can profitably be run is very good to know. The fact that people happily pay more than the inference costs means what we have now is sustainable. Maybe Anthropic of OpenAI will go out of business or something, but the weights have been calculated already, so someone will be able to offer that service going forward.

Re: Are OpenAI and Anthropic losing money on inference?

#48

Post by lkjdsklf »

JCM9 wrote: post #4

These articles (of which there are many) all make the same basic accounting mistakes. You have to include all the costs associated with the model, not just inference compute. This article is like saying an apartment complex isn’t “losing money” because the monthly rents cover operating costs but ignoring the cost of the building. Most real estate developments go bust because the developers can’t pay the mortgage paym…

It’s fun to work backwards, but i was listening to a podcast where the journalists were talking about a dinner that Sam Altman had.

This question came up and Sam said they were profitable if you exclude training and the COO corrected him

So at least for OpenAI, the answer is “no”

They did say it was close

And that’s if you exclude training costs which is kind of absurd because it’s not like you can stop training

Re: Are OpenAI and Anthropic losing money on inference?

#49

Post by Aurornis »

uludag wrote: post #44

Another comment mentioned the cost associated with the model. Setting that aside, wouldn't we also need to include all of the systems around the inference? I can imagine significant infrastructure and engineering needs around all of these various services, along with the work needed to keep these systems up and running. Or are these costs just insignificant compared to inference?

All incremental costs should be included. If adding each 100,000 new customers requires 1 extra engineer you would include that. We don’t know those exact numbers though and the ratio is probably much higher than my example numbers. Inference costs likely dominate.

Re: Are OpenAI and Anthropic losing money on inference?

#50

Post by conradev »

JCM9 wrote: post #4

These articles (of which there are many) all make the same basic accounting mistakes. You have to include all the costs associated with the model, not just inference compute. This article is like saying an apartment complex isn’t “losing money” because the monthly rents cover operating costs but ignoring the cost of the building. Most real estate developments go bust because the developers can’t pay the mortgage paym…

I found Dario’s explanation pretty compelling:

https://x.com/FinHubIQ/status/1960540489876410404

the short of it: if you do the accounting on a per-model basis, it looks much better

Post reply on HN