Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Are OpenAI and Anthropic losing money on inference?

martinalderson.com

Post reply on HN
Print view View original

471–480 of 495 posts

Re: Are OpenAI and Anthropic losing money on inference?

#471

Post by achenet »

Tossrock wrote: post #372
achenet wrote: post #326

Earlier quoted context omitted.

He makes money by convincing people to buy OpenAI stock. If OpenAI goes down tomorrow, he will be just fine. His incentive is to sell the stock, not actually build and run a profitable business. Look at Adam Neumann as an example of how to lose billions of investor dollars and still walk out of the ensuing crash with over a billion. https://en.wikipedia.org/wiki/Adam_Neumann His strategy is to sell OpenAI stock like…

Altman doesn't have any stock. He's playing a game at a level people caught up on "capitalism bad" can't even conceptualize.

I'm more "capitalism good" (8 billion people on earth, 7 billion can read, 5 billion have internet, and almost no one dies in childbirth anymore in rich countries, which is several billion people), but that is really interesting that he has no stock and just gets salary.

I guess if other people buying stock in your company is what enables you to have a super high salary (+ benefits like company plane, etc), you are still kinda selling stock though, and honestly, having considered the "start a random software company aligned with the present trend (so ~2015 DevOps/Cloud, 2020 cryptocurrency/blockchain, 2024 AI/ML), pay myself a million dollar a year salary and close shop after 5 years because 'no market lol'" route to riches myself, I still wouldn't consider Altman to be completely free of perverse incentives here :)

Still, very glad you pointed that out, thanks for sharing that information ^^

Re: Are OpenAI and Anthropic losing money on inference?

#472

Post by topaz0 »

ascorbic wrote: post #404
topaz0 wrote: post #392

Earlier quoted context omitted.

Yes, and the comment you first replied to was about the state/viability of the industry as a whole. If users can't make money from this "transformative technology", even when the provider is in the stage of burning money for the sake of growth, that sort of tells against it turning into a trillion dollar industry or whatever the hype claims.

The point is that the providers aren't burning money by subsidising inference costs. On the contrary, if this article is to believed they're charging healthy margins on it. So there are two answers: for the model providers, it's because they're spending it all on training the next model. For the API users, it's because they're spending it all on expensive API usage.

The point is that the margins aren't "healthy" for the industry if their users can't be profitable, because if that's the case, the users will all go out of business, and the providers will stop being able to charge anybody anything, margin or no.

Re: Are OpenAI and Anthropic losing money on inference?

#473

Post by ath3nd »

wahnfrieden wrote: post #310
ath3nd wrote: post #151

Yes they are, they are deeply deeply unprofitable and that's why they need endless investments to prop them up. That's why Microsoft is not doing the deal with OpenAI, that's why Claude was fiddling with token limits just a couple of weeks ago. It's a huge bubble, and the only winner at this moment is Nvidia.

Citation? Investments aren't evidence of unprofitability in inference

> Citation? Investments aren't evidence of unprofitability in inference

Does a quote from their CEO help you?

"Anthropic CEO Dario Amodei has indicated that the individual models that the company creates are profitable, even if one includes the cost of training them. But given how Anthropic keeps training new models, it reports overall losses at a company level."

https://officechai.com/ai/each-individual-ai-model-can-alrea...

That good enough for you?

It's also obvious by the fact they are starting to train on your data, like the grifters they are: https://www.theverge.com/anthropic/767507/anthropic-user-dat...

They are struggling to do anything to survive, with a crushing debt. Soon Ads :)

Re: Are OpenAI and Anthropic losing money on inference?

#474

Post by lumost »

utyop22 wrote: post #469
Jaxkr wrote: post #449

Earlier quoted context omitted.

I think some of the power user demand is fairly inelastic. I’ve seen developers who are allergic to spending money happily drop $200/mo on those new Claude subscriptions.

Yeah but if you push the price up, given that many users will cancel their subscriptions you will end up with still a tiny market segment relative to what is necessary, in revenues, to justify the valuations purported.

It's a tricky one, there is also a lot of push right now to use AI so developers are incentivized to drop money on subscriptions. I'd have difficulty justifying 1k/month for smaller shops - but corporations will be different. If the average engineer is just 20% more productive, then that is a 30-60k value to the company.

I don't have difficulty getting to a 20% productivity gain with AI just from automating the tasks I procrastinate on or can't focus on. Likewise the ability to code a prototype overnight/over the weekend is a reasonable extension of practical working hours.

The challenge I do see is that fully AI generated code bases devolve into slop pretty fast. The productivity cutoffs are much lower compared to human engineers.

Re: Are OpenAI and Anthropic losing money on inference?

#475

Post by vonneumannstan »

ewoodrich wrote: post #446
vonneumannstan wrote: post #321

Earlier quoted context omitted.

I suspect we've already reached the point with models at the GPT5 tier where the average person will no longer recognize improvements and this model can be slightly improved at slow intervals and indeed run for years. Meanwhile research grade models will still need to be trained at massive cost to improve performance on relatively short time scales.

I may not qualify as an "average user" but I shudder imagining being stuck using a 1+ yr stale model for development given my experiences using a newer framework than what was available during training. Passing in docs usually helps, but I've had some incredibly aggravating experiences where a model just absolutely cannot accept their "mental mode" is incorrect and that they need to forget the tens of thousands of li…

>Passing in docs usually helps, but I've had some incredibly aggravating experiences where a model just absolutely cannot accept their "mental mode" is incorrect and that they need to forget the tens of thousands of lines of out of date example code they've ingested during training. IMO it's an under-discussed aspect of the current effectiveness of LLM development thanks to the training arms race.

I think you overestimate the amount of code turnover in 6-12 months...

Re: Are OpenAI and Anthropic losing money on inference?

#476

Post by vonneumannstan »

felipeerias wrote: post #441
vonneumannstan wrote: post #321

Earlier quoted context omitted.

I suspect we've already reached the point with models at the GPT5 tier where the average person will no longer recognize improvements and this model can be slightly improved at slow intervals and indeed run for years. Meanwhile research grade models will still need to be trained at massive cost to improve performance on relatively short time scales.

The "Pro" variant of GTP-5 is probably the best model around and most people are not even aware that it exists. One reason is that as models get more capable, they also get a lot more expensive to run so this "Pro" is only available at the $200/month pro plan. At the same time, more capable models are also a lot more expensive to train. The key point is that the relationship between all these magnitudes is not linear…

>Soon we will probably arrive at a point where these huge training runs must stop, because the performance improvement does not match the huge cost increase, and because the resulting model would be so expensive to run that the market for it would be too small.

I think we're a lot more likely to get to the limit of power and compute available for training a bigger model before we get to the point where improvement stops.

Re: Are OpenAI and Anthropic losing money on inference?

#477

Post by wahnfrieden »

ath3nd wrote: post #473
wahnfrieden wrote: post #310

Earlier quoted context omitted.

Citation? Investments aren't evidence of unprofitability in inference

> Citation? Investments aren't evidence of unprofitability in inference Does a quote from their CEO help you? "Anthropic CEO Dario Amodei has indicated that the individual models that the company creates are profitable, even if one includes the cost of training them. But given how Anthropic keeps training new models, it reports overall losses at a company level." https://officechai.com/ai/each-individual-ai-model-can…

I asked about profitability of inference because that’s what this article/thread are about. Your quote is evidence that inference is profitable. Thank you.

Re: Are OpenAI and Anthropic losing money on inference?

#478

Post by simianwords »

achenet wrote: post #471
Tossrock wrote: post #372

Earlier quoted context omitted.

Altman doesn't have any stock. He's playing a game at a level people caught up on "capitalism bad" can't even conceptualize.

I'm more "capitalism good" (8 billion people on earth, 7 billion can read, 5 billion have internet, and almost no one dies in childbirth anymore in rich countries, which is several billion people), but that is really interesting that he has no stock and just gets salary. I guess if other people buying stock in your company is what enables you to have a super high salary (+ benefits like company plane, etc), you are s…

Again incorrect. He doesn’t have a super high salary.

Re: Are OpenAI and Anthropic losing money on inference?

#479

Post by hirako2000 »

CjHuber wrote: post #274
hirako2000 wrote: post #157

Earlier quoted context omitted.

That' what the buzz focused on, strange as we don't actually know what it cost them. While inference optimization is a fact and is even more impactful since training costs benefit from economics of scale.

I don't think that's strange at all, it's a much more palatable narrative for the mass who doesn't know what inference and training is and who think having conversations=training

I agree nothing surprising in that, also back then inference wasn't as much questioned as today with regards to being sold at a loss.

Re: Are OpenAI and Anthropic losing money on inference?

#480

Post by ath3nd »

wahnfrieden wrote: post #477
ath3nd wrote: post #473

Earlier quoted context omitted.

> Citation? Investments aren't evidence of unprofitability in inference Does a quote from their CEO help you? "Anthropic CEO Dario Amodei has indicated that the individual models that the company creates are profitable, even if one includes the cost of training them. But given how Anthropic keeps training new models, it reports overall losses at a company level." https://officechai.com/ai/each-individual-ai-model-can…

I asked about profitability of inference because that’s what this article/thread are about. Your quote is evidence that inference is profitable. Thank you.

Haha, profitable on some models is not the same as profitable. They are deeply unprofitable as a company and that's why they are grifting, changing plans and lowering limits, and now training on your data. Grifters gonna grift.
Post reply on HN