Earlier quoted context omitted.
[flagged]
why not?
Obviously because you are now allowed to download and share copyrighted works without permission or cost. At least that seems to be the precedent being set in court cases thus far.
Are OpenAI and Anthropic losing money on inference?
341–350 of 495 posts
Re: Are OpenAI and Anthropic losing money on inference?
#342Re: Are OpenAI and Anthropic losing money on inference?
#343Only introducing this *NVIDIA AI Released Jet-Nemotron: 53x Faster Hybrid-Architecture Language Model Series that Translates to a 98% Cost Reduction for Inference at Scale, into the conversation as it just dropped (so it is timely) and while it seems unlikely either OpenAI or Anthropic use this or a technique like it (yet or if they even can), these types of breakthroughs may introduce dramatic savings for both closed and open source inference at scale moving forward https://www.marktechpost.com/2025/08/26/nvidia-ai-released-j...
Re: Are OpenAI and Anthropic losing money on inference?
#344Earlier quoted context omitted.
> but if inference is this cheap then why aren't there multiple API providers offering models at dirt cheap prices There are multiple API providers offering models at dirt cheap prices, enough so that there is at least one well-known API provider that is an aggreggator of other API providers that offers lots of models at $0. > The only cheap-ass providers I've seen only run tiny models. Where's my cheap deepseek-R1?…
How is this possible? I imagine someone is finding some value in the prompts themselves but this cant possibly be paying for itself.
Inference is just that cheap plus they hope that you'll start using the ones they charge for as you become more used to using AI in your workflow.
Re: Are OpenAI and Anthropic losing money on inference?
#345These numbers are off. > $20/month ChatGPT Pro user: Heavy daily usage but token-limited ChatGPT Pro is $200/month and Sam Altman already admitted that OpenAI is losing money from Pro subscriptions in January 2025: "insane thing: we are currently losing money on openai pro subscriptions! people use it much more than we expected." - Sam Altman, January 6, 2025 https://xcancel.com/sama/status/1876104315296968813
I just straight up don't trust him Saying that is the equivalent of him saying "our product is really valuable! use it!"
That is my interpretation, that it's a marketing attempt. A form of "The value of our product is so good that it's losing us money. It's practically the Costco hotdog combo!".
Re: Are OpenAI and Anthropic losing money on inference?
#346Earlier quoted context omitted.
Well, only if the one training model continued to function as a going business. Their amortization window for the training cost is 2 months or so. They can't just keep that up and collect $. They have to build the next model, or else people will go to someone else.
Why two months? It was almost a year between Claude 3.5 and 4. (Not sure how much it costs to go from 3.5 to 3.7.)
Don't they need to accelerate that, though? Having a 1 year old model isn't really great, it's just tolerable.
Re: Are OpenAI and Anthropic losing money on inference?
#347I've done the modeling on this a few times and I always get to a place where inference can run at 50%+ gross margins, depending mostly on GPU depreciation and how good the host is at optimizing utilization. The challenge for the margins is whether or not you consider model training costs as part of the calculation. If model training isn't capitalized + amortized, margins are great. If they are amortized and need to b…
I have to disagree. The biggest cost is still energy consumption, water and maintenance. Not to mention, to keep up with the rivals in incredibly high tempo (so offering billions like Meta recently). Then the cost of hardware that is equal to Nvidia skyrocketing shares :) No one should dare to talk about profit yet. Now is time to grab the market, invest a lot and work hard, hopping for a future profit. The equation…
Is that not baked into the h100 rental costs?
Re: Are OpenAI and Anthropic losing money on inference?
#348Earlier quoted context omitted.
Why two months? It was almost a year between Claude 3.5 and 4. (Not sure how much it costs to go from 3.5 to 3.7.)
Even being generous, and saying it's a year, most capital expenditures depreciate over a period of 5-7 years. To state the obvious, training one model a year is not a saving grace
I don't understand why the absolute time period matters — all that matters is that you get enough time making money on inference to make up for the cost of training.
Re: Are OpenAI and Anthropic losing money on inference?
#349Earlier quoted context omitted.
I wasn’t using COGS in a GAAP sense, but rather as a synonym for unspecified “costs.” My bad. I suppose you would classify training as development and ongoing datacenter and GPU costs as actual GAAP COGS. My point was, if all you focus on is revenue and ignore the costs of creating your business and keeping it running, it’s pretty easy for any business to be “profitable.”
It’s generally useful to consider unit economy separate from whole company. If your unit economy is negative thing are very bleak. If it’s positive, your chance are going up by a lot - scaling the business amortizes fixed (non-unit) costs, such as admin and R&D, and slightly improves unit margins as well. However this does not work as well if your fixed (non-unit) cost is growing exponentially. You can’t get out of t…
It’s splitting a hair, but a pretty important hair. Does anyone think that models won’t need continuous retraining? Does anyone think models won’t continue to try to scale? Personally, I think we’re reaching diminishing returns with scaling, which is probably good because we’ve basically run out of content to train on, and so perhaps that does stop or at least slow down drastically. But I don’t see a scenario where constant retraining isn’t the norm, even if the rough amount of content we’re using for it grows only slightly.
Re: Are OpenAI and Anthropic losing money on inference?
#350Earlier quoted context omitted.
Why two months? It was almost a year between Claude 3.5 and 4. (Not sure how much it costs to go from 3.5 to 3.7.)
Don't they need to accelerate that, though? Having a 1 year old model isn't really great, it's just tolerable.
I think this is debatable as more models become good enough for more tasks. Maybe a smaller proportion of tasks will require SOTA models. On the other hand, the set of tasks people want to use LLMs for will expand along with the capabilities of SOTA models.