181–190 of 222 posts
Earlier quoted context omitted.
Well, it is currently cheaper because it is massively subsidized. That will change when subsidies stop. I don’t think it is a good argument.
No it's not. AI products are quite often subsidized. AI inference very certainly is not. There are more and more independent AI inference providers without VC backing that serve open weight models on a ~cost-plus basis that show that subsidies are not significant for AI inference.
We are slowly inching closer to the point where AI and AI products will be billed for what they cost. We are currently living in the heavily discounted world where everything subsidized to the point where a lot of it is free. It seems like they can't or won't keep that up anymore. My prediction is that whenever one of the big companies raise their prices or move features to higher tiers others will follow soon. They…
It is already bafflingly expensive. I interviewed at a place recently where they said the average dev was hitting $2k/mo in Claude code costs. That is no longer a helpful tool... it costs like ~15% of an actual dev. Even if it is helping, is it actually... making things better or building anything truly important? The issue seems way too nuanced to spend $2k/mo. Not to mention the entire tech industry floats on hype…
Earlier quoted context omitted.
We don’t know the models sizes, requirements, and optimisations, but we could take a guess using the infrastructure costs of the largest open weight alternatives that perform slightly worse. In my opinion, it’s a profitable kind of service. They probably don’t pay the public prices for the cloud GPUs though.
In my opinion it seems like a very unprofitable service propped up by investor money trying to capture market share. Or, as I would say if I were Bugs Bunny, “Duck Season”
We are slowly inching closer to the point where AI and AI products will be billed for what they cost. We are currently living in the heavily discounted world where everything subsidized to the point where a lot of it is free. It seems like they can't or won't keep that up anymore. My prediction is that whenever one of the big companies raise their prices or move features to higher tiers others will follow soon. They…
I called this last year: https://digitalseams.com/blog/the-ai-lifestyle-subsidy-is-go... . I see it as no different from the previous generation of consumer startups burning money - as Derek Thompson wrote, > ...if you woke up on a Casper mattress, worked out with a Peloton, Ubered to a WeWork, ordered on DoorDash for lunch, took a Lyft home, and ordered dinner through Postmates only to realize your partner had alrea…
The conversation around AI being cheap now started when ChatGPT launched in 2023
Earlier quoted context omitted.
> I suspect the API prices are already served at prices with profitable unit economics. There is absolutely no evidence to support this.
Some basic math supports it. A GB300 NVL72 is about $6.5 million. Lets say that you need $6 million worth of cooling and another $6 million worth of electricity. At current rates, that's 720 billion tokens worth of Claude Opus 4.7. At 100,000 tokens per second, it pays for itself in about 3 months. Obviously this is an extremely rough calculation. I can even be off by a factor of 10 and it's still a pretty good retur…
Earlier quoted context omitted.
> We are slowly inching closer to the point where AI and AI products will be billed for what they cost. I suspect the API prices are already served with profitable unit economics. The SOTA API prices are much higher than the costs for other providers to run very large open weight models. The monthly subscription plans were being offered at a discount to generate interest in these models. We're not entering a period o…
> I suspect the API prices are already served at prices with profitable unit economics. There is absolutely no evidence to support this.
Analysts like Semi-Analysis have done a lot of modeling and estimates on the topic.
But two can play this game: There is absolutely no evidence to support that API prices do not have profitable unit economics.
Earlier quoted context omitted.
> I suspect the API prices are already served at prices with profitable unit economics. There is absolutely no evidence to support this.
> There is absolutely no evidence to support this. Analysts like Semi-Analysis have done a lot of modeling and estimates on the topic. But two can play this game: There is absolutely no evidence to support that API prices do not have profitable unit economics.
Typically the burden of proof is on the one making the claim.
Earlier quoted context omitted.
You work for sandwiches and coffee, and not for a decent salary?
I get paid a decent salary for my expertise and this wild thing called remembering what happened yesterday. My point is my compute is powered by bananas and coffee, not gigawatts of energy and all of the RAM.
Thus, your compute is significantly more expensive than AI. Thankfully your taste is also part of your package deal, and is where you deliver real the value over an LLM.
This is funny, but Copilot is still an interesting case-study and (probably) failed predictor of where we are headed. We all know, and have known for a long time, that the AI labs selling dollars for a nickel are going to pull that rug, and up that price, at some point. Copilot, though, has been consistently the weakest mainstream AI coding offering. Inferior to Cursor or Windsurf at editor completions, inferior to C…
Earlier quoted context omitted.
> There is absolutely no evidence to support this. Analysts like Semi-Analysis have done a lot of modeling and estimates on the topic. But two can play this game: There is absolutely no evidence to support that API prices do not have profitable unit economics.
I'm not familiar with that analysis, its accuracy, or its evidence. I would be surprised by this given it seems like providers are still in the growth phase. Typically the burden of proof is on the one making the claim.
They have some of the best publicly available analysis on these topics. The full details and numbers are hidden behind the institutional accounts which are priced for investors (not something you sign up for personally) but they're generous with what they send out in their newsletter.
If you're not familiar with resources like this I could understand how you'd assume that the providers are hemorrhaging money on inference costs, because that is that story that gets parroted around spaces like Hacker News.
You could ignore all of that, though, and go check OpenRouter to see how much providers are selling high parameter count models. They're not entirely at the level of the SOTA models, but the biggest open weight models are not that far behind in complexity either. They're being sold an order of magnitude cheaper than what you pay for the APIs from the major players. We don't know exactly how big the major models are, but it's unlikely that they're more than 10X more compute intensive from the leaks we do have.