Are OpenAI and Anthropic losing money on inference? • Hacker News Board

Re: Are OpenAI and Anthropic losing money on inference?

#291

Post by dcre » Thu, Aug 28, 2025, 6:04 PM UTC

827a wrote: post #256

dcre wrote: post #219

Earlier quoted context omitted.

ICYMI, Amodei said the same in much greater detail: "If you consider each model to be a company, the model that was trained in 2023 was profitable. You paid $100 million, and then it made $200 million of revenue. There's some cost to inference with the model, but let's just assume, in this cartoonish cartoon example, that even if you add those two up, you're kind of in a good state. So, if every model was a company,…

OpenAI and Anthropic have very different customer bases and usage profiles. I'd estimate a significantly higher percentage of Anthropic's tokens are paid by the customer than OpenAI's. The ChatGPT free tier is magnitudes more popular than Claude's free tier, and Anthropic in all likelihood does a higher percentage of API business versus consumer business than OpenAI does. In other words, its possible this story is co…

Good point, very possible that Altman is excluding free tier as a marketing cost even if it loses more than they make on paid customers. On the other hand they may be able to cut free tier costs a lot by having the model router send queries to gpt-5-mini where before they were going to 4o.

Re: Are OpenAI and Anthropic losing money on inference?

#292

Post by WhitneyLand » Thu, Aug 28, 2025, 6:05 PM UTC

Model context limits are not “artificial” as claimed.

The largest context window a model can offer at a given quality level depends on the context size the model was pretrained with as well as specific fine tuning techniques.

It’s not simply a matter of considering increased costs.

Re: Are OpenAI and Anthropic losing money on inference?

#293

Post by johnsmith1840 » Thu, Aug 28, 2025, 6:08 PM UTC

noodletheworld wrote: post #145

Huh. I feel oddly skeptical about this article; I can't specifically argue the numbers, since I have no idea, but... there are some decent open source models; they're not state of the art, but if inference is this cheap then why aren't there multiple API providers offering models at dirt cheap prices? The only cheap-ass providers I've seen only run tiny models. Where's my cheap deepseek-R1? Surely if its this cheap,…

Another giant problem with this article is we have no idea the optimizations used on their end. There are some widly complex optimizations these large AI companies use.

What I'm trying to say is that hosting your own model is in an entierly different leauge than the pros.

If we account for error in article implies higher cost I would argue it would return back to profit directly because how advanced optimization of infer3nce has become.

If actual model intelligence is not a moat (looking likely this is true) the real sauce of profitable AI companies is advanced optimizations across the entire stack.

Openai is NEVER going to release their specialized kernels, routing algos, quanitizations or model comilation methods. These are all really hard and really specific.

Re: Are OpenAI and Anthropic losing money on inference?

#294

Post by johnsmith1840 » Thu, Aug 28, 2025, 6:10 PM UTC

gitremote wrote: post #45

These numbers are off. > $20/month ChatGPT Pro user: Heavy daily usage but token-limited ChatGPT Pro is $200/month and Sam Altman already admitted that OpenAI is losing money from Pro subscriptions in January 2025: "insane thing: we are currently losing money on openai pro subscriptions! people use it much more than we expected." - Sam Altman, January 6, 2025 https://xcancel.com/sama/status/1876104315296968813

Losing money on o1-pro. That makes sense and also why they axed that entire class of models.

Every o1-pro and o1-preview inference was a normal inference times how many replica paths they made.

Re: Are OpenAI and Anthropic losing money on inference?

#295

Post by resters » Thu, Aug 28, 2025, 6:13 PM UTC

Consider some of the scaling properties of frontier cloud LLMs:

1) routing: traffic can be routed to smaller, specialized, or quantized models

2) GPU throughput vs latency: both parameters can be tuned and adjusted based on demand. What seems like lots of deep "thinking" might just be trickling the inference over less GPU resources for longer.

3) caching

Re: Are OpenAI and Anthropic losing money on inference?

#296

Post by twoodfin » Thu, Aug 28, 2025, 6:13 PM UTC

827a wrote: post #247

simonw wrote: post #213

https://www.axios.com/2025/08/15/sam-altman-gpt5-launch-chat... quotes Sam Altman saying: > Most of what we're building out at this point is the inference [...] We're profitable on inference. If we didn't pay for training, we'd be a very profitable company.

From the latest NYT Hard Fork podcast [1]. The hosts were invited to a dinner hosted by Sam, where Sam said "we're profitable if we remove training from the equation", they report he turned to Lightcap (COO) and asked "right?" and Lightcap gave an "eeekk we're close". They aren't yet profitable even just on inference, and its possible Sam didn't know that until very recently. [1] https://www.nytimes.com/2025/08/22/po…

“We’re not profitable even if we discount training costs.”

and

“Inference revenue significantly exceeds inference costs.”

are not incompatible statements.

So maybe only the first part of Sam’s comment was correct.

Re: Are OpenAI and Anthropic losing money on inference?

#297

Post by Der_Einzige » Thu, Aug 28, 2025, 6:13 PM UTC

thegeomaster wrote: post #216

ankit219 wrote: post #160

This seems very very far off. From the latest reports, anthropic has a gross margin of 60%. It came out in their latest fundraising story. From that one The Information report, it estimated OpenAI's GM to be 50% including free users. These are gross margins so any amortization or model training cost would likely come after this. Then, today almost every lab uses methods like speculative decoding and caching which red…

Are you saying that you think Sonnet 4 has 100B-200B _active_ params? And that Opus has 2T active? What data are you basing these outlandish assumptions on?

Not everyone uses MoE architectures. It's not outlandish at all...

Re: Are OpenAI and Anthropic losing money on inference?

#298

Post by RhythmFox » Thu, Aug 28, 2025, 6:14 PM UTC

KallDrexx wrote: post #249

Since DeepSeek R1 is open weight, wouldn't it be better to validate the napkin math to validate how many realistic LLM full inferences can be done on a single H100 in a time period, and calculate the token cost of that? Without having in depth knowledge of the industry, the margin difference between input and output tokens is very odd to me between your napkin math and the R1 prices. That's very important as any reas…

I am so glad someone else called this out, I was reading the napkin math portions and struggling to see how the numbers really worked out and I think you hit the nail on the head. The author is assuming 'essentially free' input token cost and extrapolating in a business model that doesn't seem to connect directly to any claimed 'usefulness'. I think the bias on this is stated in the beginning of the article clearly as the author assumes 'given how useful the current models are...'. That is not a very scientific starting point and I think it leads to reasoning errors within the business model he posits here.

There were some oddities with the numbers themselves as well but I think it was all within rounding, though it would have been nice for the author to spell it out when he rounded some important numbers (~s don't tell me a whole lot).

TL;DR I totally agree, there are some napkin math issues going on here that make this pretty hard to see as a very useful stress test of cost.

Re: Are OpenAI and Anthropic losing money on inference?

#299

Post by colinsane » Thu, Aug 28, 2025, 6:14 PM UTC

LZ_Khan wrote: post #265

dcre wrote: post #219

Earlier quoted context omitted.

ICYMI, Amodei said the same in much greater detail: "If you consider each model to be a company, the model that was trained in 2023 was profitable. You paid $100 million, and then it made $200 million of revenue. There's some cost to inference with the model, but let's just assume, in this cartoonish cartoon example, that even if you add those two up, you're kind of in a good state. So, if every model was a company,…

Also Amodei has an assumption that a 100m model will make 200m of revenue but a 1B model will make 2B of revenue. Does that really hold up? There's no phenomenon that prevents them from only making 200m of revenue off a $1B model.

> So, there'll be a one-time, 'Oh man, we spent a lot of money and we didn't get anything for it.'

Re: Are OpenAI and Anthropic losing money on inference?

#300

Post by Der_Einzige » Thu, Aug 28, 2025, 6:15 PM UTC

WhitneyLand wrote: post #292

Model context limits are not “artificial” as claimed. The largest context window a model can offer at a given quality level depends on the context size the model was pretrained with as well as specific fine tuning techniques. It’s not simply a matter of considering increased costs.

Context extension methods exist and work. Please educate yourself about these rather than confidentially saying wrong things.