Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Are OpenAI and Anthropic losing money on inference?

martinalderson.com

Post reply on HN
Print view View original

221–230 of 495 posts

Re: Are OpenAI and Anthropic losing money on inference?

#221

Post by ffsm8 »

gmd63 wrote: post #208
phillipcarter wrote: post #152

Earlier quoted context omitted.

Uhhh, I'm pretty sure DeepSeek shook the industry because of a 14x reduction in training cost, not inference cost. We also don't know the per-token cost for OpenAI and Anthropic models, but I would be highly surprised if it was significantly more expensive than open models anyone can use and run themselves. It's not like they're also not investing in inference research.

DeepSeek was trained with distillation. Any accurate estimate of training costs should include the training costs of the model that it was distilling.

That makes the calculation nonsensical, because if you go there... you'd also have to include all energy used in producing the content the other model providers used. So now suddenly everyones devices on which they wrote comments on social media, pretty much all servers to have ever served a request to open AI/Google/anthropics bots etc pp

Seriously, that claim was always completely disingenuous

Re: Are OpenAI and Anthropic losing money on inference?

#222

Post by dragonwriter »

ac29 wrote: post #187
jsnell wrote: post #166

Earlier quoted context omitted.

> why aren't there multiple API providers offering models at dirt cheap prices? There are. Basically every provider's R1 prices are cheaper than estimated by this article. https://artificialanalysis.ai/models/deepseek-r1/providers

The cheapest provider in your link charges 460x more for input tokens than the article estimates.

> The cheapest provider in your link charges 460x more for input tokens than the article estimates.

The article estinates $0.003 per million input tokens, the cheapest on the list is $0.46 per million. The ratio is 120×, not 460×.

OTOH, all of the providers are far below the estimated $3.08 cost per million output tokens

Re: Are OpenAI and Anthropic losing money on inference?

#223

Post by drob518 »

simonw wrote: post #213

https://www.axios.com/2025/08/15/sam-altman-gpt5-launch-chat... quotes Sam Altman saying: > Most of what we're building out at this point is the inference [...] We're profitable on inference. If we didn't pay for training, we'd be a very profitable company.

Which is like saying, “If all we did is charge people money and didn’t have any COGS, we’d be a very profitable company.” That’s a truism of every business and therefore basically meaningless.

Re: Are OpenAI and Anthropic losing money on inference?

#224

Post by aeternum »

simonw wrote: post #213

https://www.axios.com/2025/08/15/sam-altman-gpt5-launch-chat... quotes Sam Altman saying: > Most of what we're building out at this point is the inference [...] We're profitable on inference. If we didn't pay for training, we'd be a very profitable company.

This can be technically true without being actually true.

IE OpenAI invests in Cursor/Windsurf/Startups that give away credits to users and make heavy use of inference API. Money flows back to OpenAI then OpenAI sends it back to those companies via credits/investment $.

It's even more circular in this case because nvidia is also funding companies that generate significant inference.

It'll be quite difficult to figure out whether it's actually profitable until the new investment dollars start to dry up.

Re: Are OpenAI and Anthropic losing money on inference?

#225

Post by asabla »

ugh123 wrote: post #220
simonw wrote: post #213

https://www.axios.com/2025/08/15/sam-altman-gpt5-launch-chat... quotes Sam Altman saying: > Most of what we're building out at this point is the inference [...] We're profitable on inference. If we didn't pay for training, we'd be a very profitable company.

That might be the case, but inference times have only gone up since GPT-3 (GPT-5 is regularly 20+ seconds for me).

And by GPT-5 you mean through their API? Directly through Azure OpenAI services? or are you talking about ChatGPT set to using GPT-5.

All of these alternatives means different things when you say it takes +20 seconds for a full response.

Re: Are OpenAI and Anthropic losing money on inference?

#226

Post by tsunamifury »

otterley wrote: post #124
tsunamifury wrote: post #117

Earlier quoted context omitted.

As someone who has been taking the largest part of Google and facebooks ad wallet share away, Let me tell you something. Advertising is now a very very locked in market and will take over a decade to shift even a significant minority it into OpenAIs hands. This is not likely the first or even second monetization strategy imo. But I’m happy to be wrong.

> As someone who has been taking the largest part of Google and facebooks ad wallet share away Can you elaborate? You’ve sparked my curiosity.

There are two companies gaining significant wallet share: Amazon and TikTok. Of those only one is taking a significant early share of both Google and Facebook.

Re: Are OpenAI and Anthropic losing money on inference?

#227

Post by Onavo »

qrios wrote: post #203

For sure an interesting calculation. Only one remark from someone with GPU metal experience: > But compute becomes the bottleneck in certain scenarios. With long context sequences, attention computation scales quadratically with sequence length. Even if the statement about quadratically scales is right, the bottleneck we are talking about is somewhere north by factor 1000. If 10k cores do only simple matrix operation…

What do you mean by collision?

Re: Are OpenAI and Anthropic losing money on inference?

#228

Post by [deleted] »

drob518 wrote: post #223
simonw wrote: post #213

https://www.axios.com/2025/08/15/sam-altman-gpt5-launch-chat... quotes Sam Altman saying: > Most of what we're building out at this point is the inference [...] We're profitable on inference. If we didn't pay for training, we'd be a very profitable company.

Which is like saying, “If all we did is charge people money and didn’t have any COGS, we’d be a very profitable company.” That’s a truism of every business and therefore basically meaningless.

[deleted]

Re: Are OpenAI and Anthropic losing money on inference?

#229

Post by dcre »

drob518 wrote: post #223
simonw wrote: post #213

https://www.axios.com/2025/08/15/sam-altman-gpt5-launch-chat... quotes Sam Altman saying: > Most of what we're building out at this point is the inference [...] We're profitable on inference. If we didn't pay for training, we'd be a very profitable company.

Which is like saying, “If all we did is charge people money and didn’t have any COGS, we’d be a very profitable company.” That’s a truism of every business and therefore basically meaningless.

The Amodei quote in my other reply explains why this is wrong. The point is not to compare the training of the current model to inference on the current model. The thing that makes them lose so much money is that they are training the next model while making back their training cost on the current model. So it's not COGS at all.

Re: Are OpenAI and Anthropic losing money on inference?

#230

Post by svachalek »

booi wrote: post #198
dragonwriter wrote: post #177

Earlier quoted context omitted.

> but if inference is this cheap then why aren't there multiple API providers offering models at dirt cheap prices There are multiple API providers offering models at dirt cheap prices, enough so that there is at least one well-known API provider that is an aggreggator of other API providers that offers lots of models at $0. > The only cheap-ass providers I've seen only run tiny models. Where's my cheap deepseek-R1?…

you can also run deepseek for free on a modestly sized laptop

You're probably thinking of what ollama labels "deepseek" which is not in fact deepseek, but other models with some deepseek distilled into them.
Post reply on HN