Live data from Hacker News

Are OpenAI and Anthropic losing money on inference?

martinalderson.com

111–120 of 495 posts

Re: Are OpenAI and Anthropic losing money on inference?

#111
post #4

These articles (of which there are many) all make the same basic accounting mistakes. You have to include all the costs associated with the model, not just inference compute. This article is like saying an apartment complex isn’t “losing money” because the monthly rents cover operating costs but ignoring the cost of the building. Most real estate developments go bust because the developers can’t pay the mortgage paym…

The model is like a house. It can be upgraded. And it can be sold. Think of the model as an investment.

> Think of the model as an investment.

Exactly, or a factory.

Re: Are OpenAI and Anthropic losing money on inference?

#112
post #98

The math on the input tokens is definitely wrong. It claims each instance (8 GPUs) can handle 1.44 million tokens/sec of input. Let's check that out. 1.44e6 tokens/sec * 37e9 bytes/token / 3.3e12 bytes/sec/GPU = ~16,000 GPUs And that's assuming a more likely 1 byte per parameter. So the article is only off by a factor of at least 1,000. I didn't check any of the rest of the math, but that probably has some impact on…

37 billion bytes per token?

Edit: Oh assuming this is an estimate based on the model weights moving fromm HBM to SRAM, that's not how transformers are applied to input tokens. You only have to do move the weights for every token during generation, not during "prefill". (And actually during generation you can use speculative decoding to do better than this roofline anyways).

Re: Are OpenAI and Anthropic losing money on inference?

#113

This kind of presumes you're just cranking out inference non-stop 24/7 to get the estimated price, right? Or am I misreading this? In reality, presumably they have to support fast inference even during peak usage times, but then the hardware is still sitting around off of peak times. I guess they can power them off, but that's a significant difference from paying $2/hr for an all-in IaaS provider. I'm also not sure w…

That's why they have the batch tier: https://platform.openai.com/docs/guides/batch

Re: Are OpenAI and Anthropic losing money on inference?

#114

These numbers are off. > $20/month ChatGPT Pro user: Heavy daily usage but token-limited ChatGPT Pro is $200/month and Sam Altman already admitted that OpenAI is losing money from Pro subscriptions in January 2025: "insane thing: we are currently losing money on openai pro subscriptions! people use it much more than we expected." - Sam Altman, January 6, 2025 https://xcancel.com/sama/status/1876104315296968813

That doesn't seem compatible with what he stated more recently:

> We're profitable on inference. If we didn't pay for training, we'd be a very profitable company.

Source: https://www.axios.com/2025/08/15/sam-altman-gpt5-launch-chat...

His possible incentives and the fact OpenAI isn't a public company simply make it hard for us to gauge which of these statements is closer to the truth.

Re: Are OpenAI and Anthropic losing money on inference?

#115

From https://www.theverge.com/command-line-newsletter/759897/sam-... , Sam Altman said: > “If we didn’t pay for training, we’d be a very profitable company.”

Exactly. All of the claims that OpenAI is losing money on every request are wrong. OpenAI hasn’t even unlocked all of their possible revenue opportunities from the free tier such as ads (like Google search), affiliate links, and other services. There’s also a lot of comments in this thread who want LLM companies to fail for different reasons, so they’re projecting that wish on to imagined unit economics. I’m having f…

No, the argument is that Uber was going to lose money hand over fist until all of the alternatives were starved to death, then raise prices infinitely.

Re: Are OpenAI and Anthropic losing money on inference?

#116
post #98

The math on the input tokens is definitely wrong. It claims each instance (8 GPUs) can handle 1.44 million tokens/sec of input. Let's check that out. 1.44e6 tokens/sec * 37e9 bytes/token / 3.3e12 bytes/sec/GPU = ~16,000 GPUs And that's assuming a more likely 1 byte per parameter. So the article is only off by a factor of at least 1,000. I didn't check any of the rest of the math, but that probably has some impact on…

> 37e9 bytes/token This doesn't quite sound right...isn't a token just a few characters?

[deleted]

Re: Are OpenAI and Anthropic losing money on inference?

#117

From https://www.theverge.com/command-line-newsletter/759897/sam-... , Sam Altman said: > “If we didn’t pay for training, we’d be a very profitable company.”

Exactly. All of the claims that OpenAI is losing money on every request are wrong. OpenAI hasn’t even unlocked all of their possible revenue opportunities from the free tier such as ads (like Google search), affiliate links, and other services. There’s also a lot of comments in this thread who want LLM companies to fail for different reasons, so they’re projecting that wish on to imagined unit economics. I’m having f…

As someone who has been taking the largest part of Google and facebooks ad wallet share away, Let me tell you something.

Advertising is now a very very locked in market and will take over a decade to shift even a significant minority it into OpenAIs hands. This is not likely the first or even second monetization strategy imo.

But I’m happy to be wrong.

Re: Are OpenAI and Anthropic losing money on inference?

#118
post #114

These numbers are off. > $20/month ChatGPT Pro user: Heavy daily usage but token-limited ChatGPT Pro is $200/month and Sam Altman already admitted that OpenAI is losing money from Pro subscriptions in January 2025: "insane thing: we are currently losing money on openai pro subscriptions! people use it much more than we expected." - Sam Altman, January 6, 2025 https://xcancel.com/sama/status/1876104315296968813

That doesn't seem compatible with what he stated more recently: > We're profitable on inference. If we didn't pay for training, we'd be a very profitable company. Source: https://www.axios.com/2025/08/15/sam-altman-gpt5-launch-chat... His possible incentives and the fact OpenAI isn't a public company simply make it hard for us to gauge which of these statements is closer to the truth.

> That doesn't seem compatible with what he stated more recently:

Profitable on inference doesn't mean they aren't losing money on pro plans. What's not compatible?

The API requests are likely making more money.

Re: Are OpenAI and Anthropic losing money on inference?

#119

Earlier quoted context omitted.

It’s fun to work backwards, but i was listening to a podcast where the journalists were talking about a dinner that Sam Altman had. This question came up and Sam said they were profitable if you exclude training and the COO corrected him So at least for OpenAI, the answer is “no” They did say it was close And that’s if you exclude training costs which is kind of absurd because it’s not like you can stop training

There’s no mention of that in this article about it: https://archive.is/wZslL They quote him as saying inference is profitable and end it at that. Are you saying that the COO corrected him at the dinner, or on the podcast? Which podcast was it?

From a journalist at the dinner:

“I think that tends to end poorly because as demand for your service grows, you lose more and more money. Sam Altman actually addressed this at dinner. He was asked basically, are you guys losing money every time someone uses ChatGPT?

And it was funny. At first, he answered, no, we would be profitable if not for training new models. Essentially, if you take away all the stuff, all the money we're spending on building new models and just look at the cost of serving the existing models, we are sort of profitable on that basis.

And then he looked at Brad Lightcap, who is the COO, and he sort of said, right? And Brad kind of like squirmed in his seat a little bit and was like, well, we're pretty close.

We're pretty close. We're pretty close.

So to me, that suggests that there is still some, maybe small negative unit economics on the usage of ChatGPT. Now, I don't know whether that's true for other AI companies, but I think at some point, you do have to fix that because as we've seen for companies like Uber, like MoviePass, like all these other sort of classic examples of companies that were artificially subsidizing the cost of the thing that they were providing to consumers, that is not a recipe for long-term success.”

From Hard Fork: Is This an A.I. Bubble? + Meta’s Missing Morals + TikTok Shock Slop, Aug 22, 2025

Re: Are OpenAI and Anthropic losing money on inference?

#120
post #12

This kinda tracks with the latest estimate of power usage of llm inference published by google https://news.ycombinator.com/item?id=44972808 . If inference isnt that power hungry like people thought, they must be able to make good money from those subscriptions.

> power hungry like people thought

The only people who thought this were non-practitioners.

Post reply on HN