Are OpenAI and Anthropic losing money on inference?

Post by andai » Thu, Aug 28, 2025, 4:03 PM UTC

phillipcarter wrote: post #152

sc68cal wrote: post #125

This whole article is built off using DeepSeek R1, which is a huge premise that I don't think is correct. DeepSeek is much more efficient and I don't think it's a valid way to estimate what OpenAI and Anthropic's costs are. https://www.wheresyoured.at/deep-impact/ Basically, DeepSeek is _very_ efficient at inference, and that was the whole reason why it shook the industry when it was released.

Uhhh, I'm pretty sure DeepSeek shook the industry because of a 14x reduction in training cost, not inference cost. We also don't know the per-token cost for OpenAI and Anthropic models, but I would be highly surprised if it was significantly more expensive than open models anyone can use and run themselves. It's not like they're also not investing in inference research.

Post by baxtr » Thu, Aug 28, 2025, 4:05 PM UTC

phillipcarter wrote: post #152

sc68cal wrote: post #125

This whole article is built off using DeepSeek R1, which is a huge premise that I don't think is correct. DeepSeek is much more efficient and I don't think it's a valid way to estimate what OpenAI and Anthropic's costs are. https://www.wheresyoured.at/deep-impact/ Basically, DeepSeek is _very_ efficient at inference, and that was the whole reason why it shook the industry when it was released.

Uhhh, I'm pretty sure DeepSeek shook the industry because of a 14x reduction in training cost, not inference cost. We also don't know the per-token cost for OpenAI and Anthropic models, but I would be highly surprised if it was significantly more expensive than open models anyone can use and run themselves. It's not like they're also not investing in inference research.

Post by booi » Thu, Aug 28, 2025, 4:07 PM UTC

andai wrote: post #191

phillipcarter wrote: post #152

Earlier quoted context omitted.

Uhhh, I'm pretty sure DeepSeek shook the industry because of a 14x reduction in training cost, not inference cost. We also don't know the per-token cost for OpenAI and Anthropic models, but I would be highly surprised if it was significantly more expensive than open models anyone can use and run themselves. It's not like they're also not investing in inference research.

Isn't training cost a function of inference cost? From what I gathered, they reduced both. I remember seeing lots of videos at the time explaining the details, but basically it came down to the kind of hardware-aware programming that used to be very common. (Although they took it to the next level by using undocumented behavior to their advantage.)

Post by pakitan » Thu, Aug 28, 2025, 4:08 PM UTC

caminanteblanco wrote: post #110

Ok, one issue I have with this analysis is the breakdown between input and output tokens. I'm the kind of person who spend most of my chat asking questions, so I might only use 20ish input tokens per prompt, where Gemini is having to put out several hundred, which would seem to affect the economics quite a bit

Post by hirako2000 » Thu, Aug 28, 2025, 4:09 PM UTC

benreesman wrote: post #78

JCM9 wrote: post #4

These articles (of which there are many) all make the same basic accounting mistakes. You have to include all the costs associated with the model, not just inference compute. This article is like saying an apartment complex isn’t “losing money” because the monthly rents cover operating costs but ignoring the cost of the building. Most real estate developments go bust because the developers can’t pay the mortgage paym…

My observation is that Opus is chronically capacity constrained while being dramatically more expensive than any of the others. To me that more or less settles both "which one is best" and "is it subsidized". Can't be sure, but anything else defies economic gravity.