He makes money by convincing people to buy OpenAI stock. If OpenAI goes down tomorrow, he will be just fine. His incentive is to sell the stock, not actually build and run a profitable business. Look at Adam Neumann as an example of how to lose billions of investor dollars and still walk out of the ensuing crash with over a billion. https://en.wikipedia.org/wiki/Adam_Neumann His strategy is to sell OpenAI stock like…
Altman doesn't have any stock. He's playing a game at a level people caught up on "capitalism bad" can't even conceptualize.
Holy shit you are right. He owns no equity and just gets a salary. I have no idea about the game he’s playing.
Agree that the writeup is very wrong, especially for the output tokens. Here is how anyone with enough money to allocate a small cluster of powerful GPUs can decode huge models at scale, since nearly 4 months ago, with costs of 0.2 USD/million output tokens. https://lmsys.org/blog/2025-05-05-large-scale-ep/ This has gotten significantly cheaper yet with additional code hacks since then, and with using the B200s.
You can also look at the price of opensource models on openrouter, which are a fraction of the cost of closed source models. This is a market that is heavily commoditized, so I would expect it reflect the true cost with a small margin.
If you make careful calculations and estimate the theoretical margins for inference only of most of the big open models on openrouter, the margins are typically crazy high if the openrouter providers served at scale (north of 800% for most of the large models). The high cost probably reflects salaries, investments, and amortization of other expenses like free serving or occasional partial serving occupancy. Sometimes it is hard to keep uniform high load due to other preferences of users that dont get covered at any price, eg maximal context length (which is costing output performance), latency, and time for first token, but also things like privacy guarantees, or simply switching to the next best model quickly. I have always thought that centralized inference is the real goldmine of AI because you get so much value at scale for hardly any cost.
This article's math is wrong on many fundamental levels. One of the most obvious ones is that prefill is nowhere near bandwidth bound. If you compute out the MFU the author gets it's 1.44 million input tokens per second * 37 billion active params * 2 (FMA) / 8 [GPUs per instance] = 13 Petaflops per second. That's approximately 7x absolutely peak FLOPS on the hardware. Obviously, that's impossible. There's many other…
So, bottom line, do you think it’s probable that either OpenAI or Anthropic are “losing money on inference?”
Even if it is, ignoring the biggest costs going into the product and then claiming they are profitable would be actual fraud.
Why wouldn't you factor in training? It is not like you can train once and then have the model run for years. You need to constantly improve to keep up with the competition. The lifespan of a model is just a few months at this point.
In a recent episode of Hard Fork podcast, the hosts discussed an on-the-record conversation they had with Sam Altman from OpenAI. They asked him about profitability and he claimed that they are losing money mostly because of the cost of training. But as the model advances, they will train less and less. Once you take training out of the equation he claimed they were profitable based on the cost of serving the trained…
lol. TBH I don't take anyone seriously unless they are talking about cash flows (FCFF or FCFE specifically). Who cares about expense classification - show me the money!
Google and Facebook had negative free cash flow for years early in their lives. All the good investors were lolling at the bad investors lolling at the cash they were burning.
Ok and lets compare the cost of running those products and reinvestment vs the model businesses.
FCFF = EBIT(1-t)-Reinvestment. The operating expenses of the model business are much higher - so lower EBIT.
The larger the reinvestment the larger the hole. And the longer it continues (without clear steep barriers to entry to exclude competitors in the long run) it becomes harder to justify a high valuation.
I really dislike comparisons like this - it glosses over a lot of details.
DeepSeek was trained with distillation. Any accurate estimate of training costs should include the training costs of the model that it was distilling.
That makes the calculation nonsensical, because if you go there... you'd also have to include all energy used in producing the content the other model providers used. So now suddenly everyones devices on which they wrote comments on social media, pretty much all servers to have ever served a request to open AI/Google/anthropics bots etc pp Seriously, that claim was always completely disingenuous
Not just energy cost, but also licensing cost of all this content…
No. In some sense, the article comes to the right conclusion haha. But it's probably >100x off on its central premise about output tokens costing more than input.
I’m pretty sure input tokens are cheap because they want to ingest the data for training later no? They want huge contexts to slice up.
Afaik all the large providers flipped the default to contractually NOT train on your data. So no, training data context size is not a factor.
In a recent episode of Hard Fork podcast, the hosts discussed an on-the-record conversation they had with Sam Altman from OpenAI. They asked him about profitability and he claimed that they are losing money mostly because of the cost of training. But as the model advances, they will train less and less. Once you take training out of the equation he claimed they were profitable based on the cost of serving the trained…
Unfortunately for those companies, their APIs are a commodity, and are very fungible. So they'll need to keep training or be replaced with whichever competitor will. This is an exercise in attrition.
I wonder if we’re reaching a point of diminishing returns with training, at least, just by scaling the data set. I mean, there’s a finite amount of information (that can be obtained reasonably) to be trained on. I think we’re already at a sizable chunk of that, not to mention the cost of naively scaling up. My guess is that the ultimate winner will be the one that figures out how to improve without massive training costs, through better algorithms, or maybe even just better hardware (i.e. neuristors). I mean, we know that at worst case, we should be able to build something with human level intelligence that takes about 20 watts to run, and is about the size of a human head, and you only need to ingest a small slice of all available information to do that. And training should only use about 3.5 MWh, total, and can be done with the same hardware that runs the model.
What I hear nobody talking about is the price elasticity of demand and how this plays into the economics of the model business.
I think some of the power user demand is fairly inelastic. I’ve seen developers who are allergic to spending money happily drop $200/mo on those new Claude subscriptions.
Yeah but if you push the price up, given that many users will cancel their subscriptions you will end up with still a tiny market segment relative to what is necessary, in revenues, to justify the valuations purported.
He makes money by convincing people to buy OpenAI stock. If OpenAI goes down tomorrow, he will be just fine. His incentive is to sell the stock, not actually build and run a profitable business. Look at Adam Neumann as an example of how to lose billions of investor dollars and still walk out of the ensuing crash with over a billion. https://en.wikipedia.org/wiki/Adam_Neumann His strategy is to sell OpenAI stock like…
Why not build a profitable business like Zucc, Bill gates, Jensen, Sergey etc? These people are way richer much more powerful.
I believe, but have no proof, that the answer is "because it's easier to sell stock in an unprofitable business than build a profitable one", although given the other comment, there's a good chance I'm wrong about this :)