Live data from Hacker News

Are OpenAI and Anthropic losing money on inference?

martinalderson.com

11–20 of 495 posts

Re: Are OpenAI and Anthropic losing money on inference?

#11
post #7
post #4

These articles (of which there are many) all make the same basic accounting mistakes. You have to include all the costs associated with the model, not just inference compute. This article is like saying an apartment complex isn’t “losing money” because the monthly rents cover operating costs but ignoring the cost of the building. Most real estate developments go bust because the developers can’t pay the mortgage paym…

I think the point isn't to argue AI companies are money printers or even that they're fairly valued, it's that at least the unit economics work out. Contrast this to something like moviepass, where they were actually losing money on each subscriber. Sure, a company that requires huge capital investments that might never be paid back isn't great either, but at least it's better than moviepass.

Unit economics needs to include the cost of the thing being sold, not just the direct cost of selling it.

Unit economics is mostly a manufacturing concept and the only reason it looks OK here is because of not really factoring in the cost of building the thing into the cost of the thing.

Someone might say I don’t understand “unit economics” but I’d simply argue applying a unit economics argument saying it’s good without including the cost of model training is abusing the concept of unit economics in a way that’s not realistic from a business/economics sense.

The model is what’s being sold. You can’t just sell “inference” as a thing with no model. Thats just selling compute, which should be high margin. The article is simply affirming that by saying yes when you’re just selling compute in micro-chunks that’s a decent margin business which is a nice analysis but not surprising.

Re: Are OpenAI and Anthropic losing money on inference?

#13
post #4

These articles (of which there are many) all make the same basic accounting mistakes. You have to include all the costs associated with the model, not just inference compute. This article is like saying an apartment complex isn’t “losing money” because the monthly rents cover operating costs but ignoring the cost of the building. Most real estate developments go bust because the developers can’t pay the mortgage paym…

(Author here). Yes I am aware of that and did mention it. However - what I wanted to push back in this article was that claude code was completely unsustainable and therefore a flash in the pan and devs aren't at risk (I know you are not saying this).

The models as is are still hugely useful, even if no further training was done.

Re: Are OpenAI and Anthropic losing money on inference?

#14
post #4

These articles (of which there are many) all make the same basic accounting mistakes. You have to include all the costs associated with the model, not just inference compute. This article is like saying an apartment complex isn’t “losing money” because the monthly rents cover operating costs but ignoring the cost of the building. Most real estate developments go bust because the developers can’t pay the mortgage paym…

> You have to include all the costs associated with the model, not just inference.

The title of the article directly says “on inference”. It’s not a mistake to exclude training costs. This is about incremental costs of inference.

Re: Are OpenAI and Anthropic losing money on inference?

#15
post #4

These articles (of which there are many) all make the same basic accounting mistakes. You have to include all the costs associated with the model, not just inference compute. This article is like saying an apartment complex isn’t “losing money” because the monthly rents cover operating costs but ignoring the cost of the building. Most real estate developments go bust because the developers can’t pay the mortgage paym…

What will be the knock on effect on us consumers?

Costs will go up to levels where people will no longer find this stuff as useful/interesting. It’s all fun and games until the subsides end.

See the recent reactions to AWS pricing on Kiro where folks had a big WTF reaction on pricing after, it appears, AWS tried to charge realistic pricing based on what this stuff actually costs.

Re: Are OpenAI and Anthropic losing money on inference?

#16
Basically- the same math as modern automated manufacturing. Super expensive and complex build-out - then a money printer once running and optimized.

I know there is lots of bearish sentiments here. Lots of people correctly point out that this is not the same math as FAANG products - then they make the jump that it must be bad.

But - my guess is these companies end up with margins better than Tesla (modern manufacturer), but less than 80%-90% of "pure" software. Somewhere in the middle, which is still pretty good.

Also - once the Nvidia monopoly gets broken, the initial build out becomes a lot cheaper as well.

Re: Are OpenAI and Anthropic losing money on inference?

#17
post #11
post #7

Earlier quoted context omitted.

I think the point isn't to argue AI companies are money printers or even that they're fairly valued, it's that at least the unit economics work out. Contrast this to something like moviepass, where they were actually losing money on each subscriber. Sure, a company that requires huge capital investments that might never be paid back isn't great either, but at least it's better than moviepass.

Unit economics needs to include the cost of the thing being sold, not just the direct cost of selling it. Unit economics is mostly a manufacturing concept and the only reason it looks OK here is because of not really factoring in the cost of building the thing into the cost of the thing. Someone might say I don’t understand “unit economics” but I’d simply argue applying a unit economics argument saying it’s good with…

But what about running Deepseek R1 or (insert other open weights model here)? There is no training cost for that.

Re: Are OpenAI and Anthropic losing money on inference?

#18

So, if this is true, OpenAI needs much better conversion rates, because they have ~15 million paying users compared to 800 million weekly active users: https://nerdynav.com/chatgpt-statistics/

Yeah but they can probably monetize them with ads.

I'm not so sure. Inserting ads into chatbot output is like inserting ads into email. People are more reluctant to tolerate that than web or YouTube ads (which are hated already).

If they insert stealth ads, then after the third sponsored bad restaurant suggestion people will stop using that feature, too.

Re: Are OpenAI and Anthropic losing money on inference?

#19
post #15

Earlier quoted context omitted.

What will be the knock on effect on us consumers?

Costs will go up to levels where people will no longer find this stuff as useful/interesting. It’s all fun and games until the subsides end. See the recent reactions to AWS pricing on Kiro where folks had a big WTF reaction on pricing after, it appears, AWS tried to charge realistic pricing based on what this stuff actually costs.

The article is answering a specific question, and has excluded this on purpose. If you have a sunk training cost you still want to know if you can at least operate profitably.

Re: Are OpenAI and Anthropic losing money on inference?

#20

So, if this is true, OpenAI needs much better conversion rates, because they have ~15 million paying users compared to 800 million weekly active users: https://nerdynav.com/chatgpt-statistics/

Yeah but they can probably monetize them with ads.

LLM generated ads.
Post reply on HN