Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Are OpenAI and Anthropic losing money on inference?

martinalderson.com

Post reply on HN
Print view View original

261–270 of 495 posts

Re: Are OpenAI and Anthropic losing money on inference?

#261

Post by pera »

dcre wrote: post #219
simonw wrote: post #213

https://www.axios.com/2025/08/15/sam-altman-gpt5-launch-chat... quotes Sam Altman saying: > Most of what we're building out at this point is the inference [...] We're profitable on inference. If we didn't pay for training, we'd be a very profitable company.

ICYMI, Amodei said the same in much greater detail: "If you consider each model to be a company, the model that was trained in 2023 was profitable. You paid $100 million, and then it made $200 million of revenue. There's some cost to inference with the model, but let's just assume, in this cartoonish cartoon example, that even if you add those two up, you're kind of in a good state. So, if every model was a company,…

Copy laundering as a service is only profitable when you discount future settlements:

https://www.reuters.com/legal/government/anthropics-surprise...

Re: Are OpenAI and Anthropic losing money on inference?

#262

Post by meshugaas »

dcre wrote: post #219
simonw wrote: post #213

https://www.axios.com/2025/08/15/sam-altman-gpt5-launch-chat... quotes Sam Altman saying: > Most of what we're building out at this point is the inference [...] We're profitable on inference. If we didn't pay for training, we'd be a very profitable company.

ICYMI, Amodei said the same in much greater detail: "If you consider each model to be a company, the model that was trained in 2023 was profitable. You paid $100 million, and then it made $200 million of revenue. There's some cost to inference with the model, but let's just assume, in this cartoonish cartoon example, that even if you add those two up, you're kind of in a good state. So, if every model was a company,…

The "model as company" metaphor makes no sense. It should actually be models are products, like a shoe. Nike spends money developing a shoe, then building it, then they sell it, and ideally those R&D costs are made up in shoe sales. But you still have to run the whole company outside of that.

Also, in Nike's case, as they grow they get better at making more shoes for cheaper. LLM model providers tell us that every new model (shoe) costs multiples more than the last one to develop. If they make 2x revenue on training, like he's said, to be profitable they have to either double prices or double users every year, or stop making new models.

Re: Are OpenAI and Anthropic losing money on inference?

#263

Post by overgard »

jsnell wrote: post #58
JCM9 wrote: post #4

These articles (of which there are many) all make the same basic accounting mistakes. You have to include all the costs associated with the model, not just inference compute. This article is like saying an apartment complex isn’t “losing money” because the monthly rents cover operating costs but ignoring the cost of the building. Most real estate developments go bust because the developers can’t pay the mortgage paym…

For the top few providers, the training is getting amortized over absurd amount of inference. E.g. Google recently mentioned that they processed 980T tokens over all surfaces in June 2025. The leaked OpenAI financial projections for 2024 showed about equal amount of money spent on training and inference. Amortizing the training per-query really doesn't meaningfully change the unit economics. > Fact remains when all c…

Assuming users accept those ads. Like, would they make it clear with a "sponsored section", or would they just try to worm it into the output? I could see a lot of potential ways that users reject the ad service, especially if it's seen to compromise the utility or correctness of the output.

Re: Are OpenAI and Anthropic losing money on inference?

#264

Post by mcpeepants »

jedberg wrote: post #205
noodletheworld wrote: post #145

Huh. I feel oddly skeptical about this article; I can't specifically argue the numbers, since I have no idea, but... there are some decent open source models; they're not state of the art, but if inference is this cheap then why aren't there multiple API providers offering models at dirt cheap prices? The only cheap-ass providers I've seen only run tiny models. Where's my cheap deepseek-R1? Surely if its this cheap,…

https://lambda.chat Deepseek R1 for free.

* distilled R1 for free

Re: Are OpenAI and Anthropic losing money on inference?

#265

Post by LZ_Khan »

dcre wrote: post #219
simonw wrote: post #213

https://www.axios.com/2025/08/15/sam-altman-gpt5-launch-chat... quotes Sam Altman saying: > Most of what we're building out at this point is the inference [...] We're profitable on inference. If we didn't pay for training, we'd be a very profitable company.

ICYMI, Amodei said the same in much greater detail: "If you consider each model to be a company, the model that was trained in 2023 was profitable. You paid $100 million, and then it made $200 million of revenue. There's some cost to inference with the model, but let's just assume, in this cartoonish cartoon example, that even if you add those two up, you're kind of in a good state. So, if every model was a company,…

Also Amodei has an assumption that a 100m model will make 200m of revenue but a 1B model will make 2B of revenue. Does that really hold up? There's no phenomenon that prevents them from only making 200m of revenue off a $1B model.

Re: Are OpenAI and Anthropic losing money on inference?

#267

Post by stephenatgmi »

Some agent startups are already feeling the squeeze — The Information reported Cursor’s gross margins hit –16% due to token costs. So even if inference is profitable for OAI/Anthropic, downstream token-hungry apps may not see the same unit economics, and that is why token-intensive agent startups like Cursor and Perplexity are taking open-source models like Qwen or other OSS-120B and post-training them to bring down inference costs.

Re: Are OpenAI and Anthropic losing money on inference?

#268

Post by noodlescb »

simonw wrote: post #213

https://www.axios.com/2025/08/15/sam-altman-gpt5-launch-chat... quotes Sam Altman saying: > Most of what we're building out at this point is the inference [...] We're profitable on inference. If we didn't pay for training, we'd be a very profitable company.

Except these tech billionaires lie the most of the time. This is still the "grow at any cost" phase, so I don't even genuinely believe he has a confident understanding of how or at what point anything will be profitable. This just strikes me as the best answer he has at the moment.

Re: Are OpenAI and Anthropic losing money on inference?

#269

Post by reliabilityguy »

whatshisface wrote: post #251
qrios wrote: post #246

Earlier quoted context omitted.

If multiple cores tries to get the same memory addresses, the MMU feeds only one core, the second one have to whait. Depends on the type of RAM, this will cost a lot of cycles. GPU MMUs can handle multiple line in parallel. But not 10k cores at the same time. The HBM is not able to transfer 3.5TByte sequencial.

Why is that? It seems like multiple cores requesting the same address would be easier for the MMU to fetch for, not harder.

It’s not that the fetching is the problem, but serving the data to many cores at the same time from a single source.

Re: Are OpenAI and Anthropic losing money on inference?

#270

Post by CjHuber »

vkou wrote: post #169
CjHuber wrote: post #148

Earlier quoted context omitted.

Does anybody really think in this current time that what a CEO says has anything to do with reality and not just with hyping up ala elon recipe

Specifically, a connected CEO in post-law America. This sort of thing used to be called fraud, but there's zero chance of criminal prosecution.

Criminal persecution? This scheme has been perfected, like what do you want to persecute. Can you say with certainty that he means it's profitable overall? What if he means it's profitable right now today it is profitable, but not yesterday or in the last week. or what if he meant if you take the mean user its profitable? so much room for interpretation, that's why there is no risk for them
Post reply on HN