Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Are OpenAI and Anthropic losing money on inference?

martinalderson.com

Post reply on HN
Print view View original

391–400 of 495 posts

Re: Are OpenAI and Anthropic losing money on inference?

#391

Post by vonneumannstan »

black_knight wrote: post #380
vonneumannstan wrote: post #321

Earlier quoted context omitted.

I suspect we've already reached the point with models at the GPT5 tier where the average person will no longer recognize improvements and this model can be slightly improved at slow intervals and indeed run for years. Meanwhile research grade models will still need to be trained at massive cost to improve performance on relatively short time scales.

Strangely, I feel GPT-5 as the opposite of an improvement over the previous models, and consider just using Claude for actual work. Also the voice mode went from really useful to useless “Absolutely, I will keep it brief and give it to you directly. …some wrong annswer… And there you have it! As simple as that!”

>Strangely, I feel GPT-5 as the opposite of an improvement over the previous models

This is almost surely wrong but my point was about GPT5 level models in general not GPT5 specifically...

Re: Are OpenAI and Anthropic losing money on inference?

#392

Post by topaz0 »

ascorbic wrote: post #241
topaz0 wrote: post #103

Earlier quoted context omitted.

That's an argument for why openai and anthropic shouldn't be profitable, but this point is about how also they don't have customers using the models to generate a profit either. Things like cursor, for example. ETA: also note the recent MIT study that found that 95% of LLM pilots at for-profit companies were not producing returns.

This article is about the model providers' costs, not API users'. Cursor etc have to pay the marked-up inference costs, so it's not surprising they can't make a profit.

Yes, and the comment you first replied to was about the state/viability of the industry as a whole. If users can't make money from this "transformative technology", even when the provider is in the stage of burning money for the sake of growth, that sort of tells against it turning into a trillion dollar industry or whatever the hype claims.

Re: Are OpenAI and Anthropic losing money on inference?

#393

Post by th0ma5 »

AJ007 wrote: post #335
vonneumannstan wrote: post #321

Earlier quoted context omitted.

I suspect we've already reached the point with models at the GPT5 tier where the average person will no longer recognize improvements and this model can be slightly improved at slow intervals and indeed run for years. Meanwhile research grade models will still need to be trained at massive cost to improve performance on relatively short time scales.

Whenever someone has complained to me about issues they are having with ChatGPT on a particular question or type of question, the first thing I do is ask them what model they are using. So far, no one has ever known offhand what model they were using, nor were not aware there are more models! If you understand there are multiple models from multiple providers, some of those models are better at certain things than ot…

This would be helpful if there was some kind of first principle at which to gauge that better or worse comparison but there isn't outside of people's value judgements like what you're offering.

Re: Are OpenAI and Anthropic losing money on inference?

#394

Post by jrflowers »

dcre wrote: post #366
citizenpaul wrote: post #304

Earlier quoted context omitted.

There a journalist ed zittron https://www.wheresyoured.at/ That is an openai skeptic. His research if correct says not only is openai unprofitable but it likely never will be. Can't be ,its various finance ratios make early uber, amazon ect look downright fiscally frugal. He is not a tech person for what that means to you.

Zitron is not a serious analyst. https://bsky.app/profile/davidcrespo.bsky.social/post/3lxale... https://bsky.app/profile/davidcrespo.bsky.social/post/3lo22k... https://bsky.app/profile/davidcrespo.bsky.social/post/3lwhhz... https://bsky.app/profile/davidcrespo.bsky.social/post/3lv2dx...

Ed Zitron: I don’t think OpenAI will become profitable

The link you posted: I think it is very plausible that it will be hard for OpenAI to become profitable

Re: Are OpenAI and Anthropic losing money on inference?

#395

Post by chillee »

This article's math is wrong on many fundamental levels. One of the most obvious ones is that prefill is nowhere near bandwidth bound.

If you compute out the MFU the author gets it's 1.44 million input tokens per second * 37 billion active params * 2 (FMA) / 8 [GPUs per instance] = 13 Petaflops per second. That's approximately 7x absolutely peak FLOPS on the hardware. Obviously, that's impossible.

There's many other issues with this article, such as assuming only 32 concurrent requests(?), only 8 GPUs per instance as opposed to the more efficient/standard prefill-decode disagg setups, assuming that attention computation is the main thing that makes models compute-bound, etc. It's a bit of an indictment of HN's understanding of LLMs that most people are bringing up issues with the article that aren't any of the fundamental misunderstandings here.

Re: Are OpenAI and Anthropic losing money on inference?

#396

Post by crote »

simianwords wrote: post #184
crote wrote: post #24

Earlier quoted context omitted.

Their assumption is that training is a fixed cost: you'll spend the same amount on training for 5 users as you will with 500 million users. Spending hundreds of millions of dollars on training when you are two guys in a garage is quite significant, but the same amount is absolutely trivial if you are planet-scale. The big question is: how will training cost develop? Best-case scenario is a one-and-done run. But we're…

They just won’t train it. They have the choice. Why do you think they will mindlessly train extremely complicated models if the numbers don’t make sense?

Because they are trying to capture the market, obviously.

Nobody is going to pay the same price for a significantly worse model. If your competitor brings out a better model at the same price point, you either a) drop your price to attract a new low-budget market, b) train a better model to retain the same high-budget market, or c) lose all your customers.

You have taken on a huge amount of VC money, and those investors aren't going to accept options A or C. What is left is option B: burn more money, build an even better model, and hope your finances last longer than the competition.

It's the classic VC-backed startup model: operate at a loss until you have killed the competition, then slowly increase prices as your customers are unable to switch to an alternative. It worked great for Uber & friends.

Re: Are OpenAI and Anthropic losing money on inference?

#397

Post by NoahZuniga »

twoodfin wrote: post #296
827a wrote: post #247

Earlier quoted context omitted.

From the latest NYT Hard Fork podcast [1]. The hosts were invited to a dinner hosted by Sam, where Sam said "we're profitable if we remove training from the equation", they report he turned to Lightcap (COO) and asked "right?" and Lightcap gave an "eeekk we're close". They aren't yet profitable even just on inference, and its possible Sam didn't know that until very recently. [1] https://www.nytimes.com/2025/08/22/po…

“We’re not profitable even if we discount training costs.” and “Inference revenue significantly exceeds inference costs.” are not incompatible statements. So maybe only the first part of Sam’s comment was correct.

I imagine that one of the largest costs for openai is the wages they pay.

Re: Are OpenAI and Anthropic losing money on inference?

#398

Post by jacurtis »

BlindEyeHalo wrote: post #303
_sword wrote: post #276

I've done the modeling on this a few times and I always get to a place where inference can run at 50%+ gross margins, depending mostly on GPU depreciation and how good the host is at optimizing utilization. The challenge for the margins is whether or not you consider model training costs as part of the calculation. If model training isn't capitalized + amortized, margins are great. If they are amortized and need to b…

Why wouldn't you factor in training? It is not like you can train once and then have the model run for years. You need to constantly improve to keep up with the competition. The lifespan of a model is just a few months at this point.

In a recent episode of Hard Fork podcast, the hosts discussed an on-the-record conversation they had with Sam Altman from OpenAI. They asked him about profitability and he claimed that they are losing money mostly because of the cost of training. But as the model advances, they will train less and less. Once you take training out of the equation he claimed they were profitable based on the cost of serving the trained foundation models to users at current prices.

Now, when he said that, his CFO corrected him and said they aren't profitable, but said "it's close".

Take that with a grain of salt, but thats a conversation from one of the big AI companies that is only a few weeks old. I suspect that it is pretty accurate that pricing is currently reasonable if you ignore training. But training is very expensive and the reason most AI companies are losing money right now.

Re: Are OpenAI and Anthropic losing money on inference?

#399

Post by steveklabnik »

Avshalom wrote: post #332
vonneumannstan wrote: post #324

Earlier quoted context omitted.

>Also, in Nike's case, as they grow they get better at making more shoes for cheaper. This is clearly the case for models as well. Training and serving inference for GPT4 level models is probably > 100x cheaper than they used to be. Nike has been making Jordan 1's for 40+ years! OpenAI would be incredibly profitable if they could live off the profit from improved inference efficiency on a GPT4 level model!

>>This is clearly the case ... probably >>OpenAI would be incredibly profitable if they could live off the profit from improved inference efficiency on a GPT4 level model! If gpt4 was basically free money at this point it's real weird that their first instinct was to cut it off after gpt5

> If gpt4 was basically free money at this point it's real weird that their first instinct was to cut it off after gpt5

People find the UX of choosing a model very confusing, the idea with 5 is that it would route things appropriately and so eliminate this confusion. That was the motivation for removing 4. But people were upset enough that they decided to bring it back for a while, at least.

Re: Are OpenAI and Anthropic losing money on inference?

#400

Post by jacurtis »

dcre wrote: post #291
827a wrote: post #256

Earlier quoted context omitted.

OpenAI and Anthropic have very different customer bases and usage profiles. I'd estimate a significantly higher percentage of Anthropic's tokens are paid by the customer than OpenAI's. The ChatGPT free tier is magnitudes more popular than Claude's free tier, and Anthropic in all likelihood does a higher percentage of API business versus consumer business than OpenAI does. In other words, its possible this story is co…

Good point, very possible that Altman is excluding free tier as a marketing cost even if it loses more than they make on paid customers. On the other hand they may be able to cut free tier costs a lot by having the model router send queries to gpt-5-mini where before they were going to 4o.

This is very true. ChatGPT has a very generous free tier. I used to pay for it, but realized I was never really hitting the limits of what is needed to pay for it.

However, at the same time, I was using Claude much less, really preferring the answers from it most of the time, and constantly being hit with limits. So guess what I did. I cancelled my OpenAI subscription and moved to Anthropic. Not only do i get Claude Code, which OpenAI really has no serious competitor for.

I still use both models but never run into problems with OpenAI, so i see no reason to pay for it.

Post reply on HN