Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Are OpenAI and Anthropic losing money on inference?

martinalderson.com

Post reply on HN
Print view View original

181–190 of 495 posts

Re: Are OpenAI and Anthropic losing money on inference?

#181

Post by mutkach »

ankit219 wrote: post #160

This seems very very far off. From the latest reports, anthropic has a gross margin of 60%. It came out in their latest fundraising story. From that one The Information report, it estimated OpenAI's GM to be 50% including free users. These are gross margins so any amortization or model training cost would likely come after this. Then, today almost every lab uses methods like speculative decoding and caching which red…

Gross margins also don't tell the whole story, we don't know how much Azure and Amazon charge for the infrastructure and we have reasons to believe they are selling it at a massive discount (Microsoft definitely does that, as follows from their agreement with OpenAI). They get the model, OpenAI gets discounted infra.

Re: Are OpenAI and Anthropic losing money on inference?

#182

Post by paulddraper »

noodletheworld wrote: post #145

Huh. I feel oddly skeptical about this article; I can't specifically argue the numbers, since I have no idea, but... there are some decent open source models; they're not state of the art, but if inference is this cheap then why aren't there multiple API providers offering models at dirt cheap prices? The only cheap-ass providers I've seen only run tiny models. Where's my cheap deepseek-R1? Surely if its this cheap,…

I would not be surprised if the operating costs are modest

But these companies also have very expensive R&D development and large upfront costs.

Re: Are OpenAI and Anthropic losing money on inference?

#183

Post by empath75 »

moduspol wrote: post #97

This kind of presumes you're just cranking out inference non-stop 24/7 to get the estimated price, right? Or am I misreading this? In reality, presumably they have to support fast inference even during peak usage times, but then the hardware is still sitting around off of peak times. I guess they can power them off, but that's a significant difference from paying $2/hr for an all-in IaaS provider. I'm also not sure w…

> In reality, presumably they have to support fast inference even during peak usage times, but then the hardware is still sitting around off of peak times. I guess they can power them off, but that's a significant difference from paying $2/hr for an all-in IaaS provider.

They can repurpose those nodes for training when they aren't being used for inference. Or if they're using public cloud nodes, just turn them off.

Re: Are OpenAI and Anthropic losing money on inference?

#184

Post by simianwords »

crote wrote: post #24
JCM9 wrote: post #4

These articles (of which there are many) all make the same basic accounting mistakes. You have to include all the costs associated with the model, not just inference compute. This article is like saying an apartment complex isn’t “losing money” because the monthly rents cover operating costs but ignoring the cost of the building. Most real estate developments go bust because the developers can’t pay the mortgage paym…

Their assumption is that training is a fixed cost: you'll spend the same amount on training for 5 users as you will with 500 million users. Spending hundreds of millions of dollars on training when you are two guys in a garage is quite significant, but the same amount is absolutely trivial if you are planet-scale. The big question is: how will training cost develop? Best-case scenario is a one-and-done run. But we're…

They just won’t train it. They have the choice.

Why do you think they will mindlessly train extremely complicated models if the numbers don’t make sense?

Re: Are OpenAI and Anthropic losing money on inference?

#185

Post by benzible »

AstroBen wrote: post #74
gitremote wrote: post #45

These numbers are off. > $20/month ChatGPT Pro user: Heavy daily usage but token-limited ChatGPT Pro is $200/month and Sam Altman already admitted that OpenAI is losing money from Pro subscriptions in January 2025: "insane thing: we are currently losing money on openai pro subscriptions! people use it much more than we expected." - Sam Altman, January 6, 2025 https://xcancel.com/sama/status/1876104315296968813

I just straight up don't trust him Saying that is the equivalent of him saying "our product is really valuable! use it!"

There's the usual issue of a CEO "talking their book" but there's also the fact that Sam has a rich, documented history of lying. That was the central issue of his firing. "Empire of AI" has a detailed account of this. He would outright tell board member A that "board member B said X", based on his knowledge of the social dynamics of the board he assumed that A and B would never talk. But they eventually figured it out, it unraveled, and they confronted him in a group. Specifically, when they confronted him about telling Ilya Sutskever that Tasha McCauley said Helen Toner should step off the board, McCauley said "I never said that" and Altman was at a loss for words for a minute before finally mumbling "Well, I thought you could have said that. I don't know."

Re: Are OpenAI and Anthropic losing money on inference?

#186

Post by empath75 »

JCM9 wrote: post #4

These articles (of which there are many) all make the same basic accounting mistakes. You have to include all the costs associated with the model, not just inference compute. This article is like saying an apartment complex isn’t “losing money” because the monthly rents cover operating costs but ignoring the cost of the building. Most real estate developments go bust because the developers can’t pay the mortgage paym…

> If the cash flow was truly healthy these companies wouldn’t need to raise money.

If this were true, the stock market would have no reason to exist.

Re: Are OpenAI and Anthropic losing money on inference?

#187

Post by ac29 »

jsnell wrote: post #166
noodletheworld wrote: post #145

Huh. I feel oddly skeptical about this article; I can't specifically argue the numbers, since I have no idea, but... there are some decent open source models; they're not state of the art, but if inference is this cheap then why aren't there multiple API providers offering models at dirt cheap prices? The only cheap-ass providers I've seen only run tiny models. Where's my cheap deepseek-R1? Surely if its this cheap,…

> why aren't there multiple API providers offering models at dirt cheap prices? There are. Basically every provider's R1 prices are cheaper than estimated by this article. https://artificialanalysis.ai/models/deepseek-r1/providers

The cheapest provider in your link charges 460x more for input tokens than the article estimates.

Re: Are OpenAI and Anthropic losing money on inference?

#190

Post by StableAlkyne »

pessimizer wrote: post #115
Aurornis wrote: post #64

Earlier quoted context omitted.

Exactly. All of the claims that OpenAI is losing money on every request are wrong. OpenAI hasn’t even unlocked all of their possible revenue opportunities from the free tier such as ads (like Google search), affiliate links, and other services. There’s also a lot of comments in this thread who want LLM companies to fail for different reasons, so they’re projecting that wish on to imagined unit economics. I’m having f…

No, the argument is that Uber was going to lose money hand over fist until all of the alternatives were starved to death, then raise prices infinitely.

Taxis sucked. Any disruptor who was willing to just... Tell people what the cost would be ahead of time without scamming them, and show up when they said they would, was going to win.

Uber (and Lyft) didn't starve the alternatives: they were already severely malnourished. Also, they found a loophole to get around the medallion system in several cities, which taxi owners used in an incredibly anticompetitive fashion to prevent new competition.

Just because Uber used a shitty business practice to deliver the killing blow doesn't mean their competition were undeserving of the loss, or that the traditional taxis weren't without a lot of shady practices.

Post reply on HN