Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Are OpenAI and Anthropic losing money on inference?

martinalderson.com

Post reply on HN
Print view View original

431–440 of 495 posts

Re: Are OpenAI and Anthropic losing money on inference?

#431

Post by dcre »

jcranmer wrote: post #416
dcre wrote: post #366

Earlier quoted context omitted.

Zitron is not a serious analyst. https://bsky.app/profile/davidcrespo.bsky.social/post/3lxale... https://bsky.app/profile/davidcrespo.bsky.social/post/3lo22k... https://bsky.app/profile/davidcrespo.bsky.social/post/3lwhhz... https://bsky.app/profile/davidcrespo.bsky.social/post/3lv2dx...

Since only the first one responds to any of Zitron's content that I've actually read, I'll respond only to that one: It's not responsive at all to Zitron's point. Zitron's broader contention is that AI tools are not profitable because the cost of AI use is too high for users to justify spending money on the output, given the quality of output. And furthermore, he argues that this basic fact is being obscured by lots…

Already in your first point you are mixing up two claims Ed also likes to mix up. The funny thing is these claims are in direct conflict with each other. There is the question of whether people find AI worth paying for given what they get. You seem to think this is in some doubt, meanwhile here are tons of people paying for it, some even begging to be allowed to pay more in order to get more. The labs have revenue growing 20% per month. So I think that version of the point is absurd on its face. (And that's exactly why my thing about the cost-quality tradeoff being real is relevant. At least we agree on the relationship between these points.)

Ed doesn’t really make that argument anymore. The more recent form of the point is: yes, clearly people are willing to pay for it, but only because the providers are burning VC money to sell it below cost. If sold at a profit, customers would no longer find it worth it. But that’s completely different from what you’re saying. And I also think that’s not true, for a few reasons: mostly that selling near cost is the simplest explanation for the similarity of prices between providers. And now recently we have both Altman and Amodei saying their companies are selling inference at a profit.

Re: Are OpenAI and Anthropic losing money on inference?

#432

Post by nl »

tick_tock_tick wrote: post #387
lawlessone wrote: post #363

Earlier quoted context omitted.

ok, but that's just one country.

I mean for AI literally the only countries involved are the USA and China. I doubt you think China is going to start respecting IP rights anytime soon.

Mistral is in France.

Re: Are OpenAI and Anthropic losing money on inference?

#433

Post by jazzyjackson »

mindwok wrote: post #428
dgfitz wrote: post #414

Earlier quoted context omitted.

> But as the model advances, they will train less and less. They sure have a lot of training to do between now and whenever that happens. Rolling back from 5 to whatever was before it is their own admission of this fact.

I think that actually proves the opposite. People wanted an old model, not a new one, indicating that for that user base they could have just... not trained a new model.

for their user base, sure

for their investors, however, they are promising a revolution

Re: Are OpenAI and Anthropic losing money on inference?

#434

Post by utyop22 »

christina97 wrote: post #322
BlindEyeHalo wrote: post #303

Earlier quoted context omitted.

Why wouldn't you factor in training? It is not like you can train once and then have the model run for years. You need to constantly improve to keep up with the competition. The lifespan of a model is just a few months at this point.

In the same way that every other startup tries to sweep R&D costs under the rug and say “yeah but the marginal unit economics have 50% gross margins, we’ll be a great business soon”.

lol.

TBH I don't take anyone seriously unless they are talking about cash flows (FCFF or FCFE specifically).

Who cares about expense classification - show me the money!

Re: Are OpenAI and Anthropic losing money on inference?

#435

Post by utyop22 »

lumost wrote: post #371
_sword wrote: post #276

I've done the modeling on this a few times and I always get to a place where inference can run at 50%+ gross margins, depending mostly on GPU depreciation and how good the host is at optimizing utilization. The challenge for the margins is whether or not you consider model training costs as part of the calculation. If model training isn't capitalized + amortized, margins are great. If they are amortized and need to b…

I wonder how much capex risk there is in this model, depreciating the GPUs over 5 years is fine if you can guarantee utilization. Losing market share might be a death sentence for some of these firms as utilization falls.

What I hear nobody talking about is the price elasticity of demand and how this plays into the economics of the model business.

Re: Are OpenAI and Anthropic losing money on inference?

#436

Post by dcre »

jcranmer wrote: post #419
dcre wrote: post #284

Earlier quoted context omitted.

I agree we should not take CEOs at their word, we have to think about whether what they're saying is more likely to be true than false given other things we know. But to trust Zitron on anything is ridiculous. He is not a source at all: he knows very little, does zero new reporting, and frequently contradicts himself in his frenzy to believe the bubble is about to pop any time now. A simple example: claiming both tha…

Those two statements are not contradictory, and thinking that they are belies a pretty fundamental misunderstanding of his basic thesis. The first statement is one about the present value of AI. The second statement is about their belief of the future value of AI.

It is not about the present and future value of AI at all. It is about the present and future value of things other than AI. Here is the full quote:

"There is nothing else after generative AI. There are no other hypergrowth markets left in tech. SaaS companies are out of things to upsell. Google, Microsoft, Amazon and Meta do not have any other ways to continue showing growth, and when the market works that out, there will be hell to pay, hell that will reverberate through the valuations of, at the very least, every public software company, and many of the hardware ones too."

I am not doing some kind of sophisticated act of interpretation here. If AI is very little of big tech revenue, and big tech are posting massive record revenue and profits every quarter, then it cannot be the case that "there is nothing left after generative AI" and they “do not have any other ways to continue showing growth” — what is left is whatever is driving all that revenue and profit growth right now!

Re: Are OpenAI and Anthropic losing money on inference?

#437

Post by trashface »

bcrosby95 wrote: post #130
caminanteblanco wrote: post #110

Ok, one issue I have with this analysis is the breakdown between input and output tokens. I'm the kind of person who spend most of my chat asking questions, so I might only use 20ish input tokens per prompt, where Gemini is having to put out several hundred, which would seem to affect the economics quite a bit

Yeah, I've noticed Chatgpt5 is very chatty. I can ask a 1 sentence question and get back 3-4 paragraphs, most of which I ignore, depending upon the task.

Switch to Robot personality

Re: Are OpenAI and Anthropic losing money on inference?

#438

Post by y1n0 »

runako wrote: post #375
oblio wrote: post #330

Earlier quoted context omitted.

> I imagine they know this and bet on AGI primacy. Just like Uber and Tesla are betting on self driving cars. I think it's been 10 years now ("any minute now").

Notably, Uber switched horses and now runs Waymos with no human drivers.

The drivers are remote, but they are still there, dropping in as needed.

Re: Are OpenAI and Anthropic losing money on inference?

#439

Post by lukev »

Things are moving too fast to meaningfully talk about unit economics here.

My consumer-level MacBook Pro can run inference locally on models that would have been state-of-the-art anywhere just a little over a year ago.

Yet, hosted inference is definitely useful and also can benefit from economies of scale.

Also: model training is expensive and the economic reasons for doing so are complex (otherwise why release open weights?)

This is an high dimensional space, and unit economics are predicated on reducing to a few meaningful dimensions.

Re: Are OpenAI and Anthropic losing money on inference?

#440

Post by [deleted] »

lukev wrote: post #439

Things are moving too fast to meaningfully talk about unit economics here. My consumer-level MacBook Pro can run inference locally on models that would have been state-of-the-art anywhere just a little over a year ago. Yet, hosted inference is definitely useful and also can benefit from economies of scale. Also: model training is expensive and the economic reasons for doing so are complex (otherwise why release open…

[deleted]
Post reply on HN