61–70 of 100 posts
The math seems off. How is 7.8 million vs 4 million 95% more expensive. Article makes good points but I doubt the numbers as they don’t add up.
Still agree with the conclusion though.
> Anthropic’s CFO testified under oath this March that the company spent $10 billion on compute and made $5 billion in revenue (Ed Zitron has the math). The labs are underwater on inference. They’re raising prices to keep the lights on. 'The labs are underwater on inference' is an absurd thing to say whilst not separating the cost of _compute_ out into training and inference.
How is training vs inference any different than other product spaces, where all the costs of bringing a product to market have to be considered for profitability? You can't just look at marginal production cost. You are still underwater if the other development costs are not being recouped by the final sales revenue. The whole commercial AI enterprise is not economically viable if the inference revenue will not cover…
Earlier quoted context omitted.
The problem with fixating on earnings as you're doing is that it's a bad metric for a growth company. COGS is much more important. What you're doing is setting it up so every growth company is terrible until they've matured into a 20 year old company. That's obviously dumb.
From what I've seen pretty much every company is limited by hardware supply, to the level where's there complaints from current customers about the speed of new customer growth is exceeding their ability to service them properly. And "growth at all costs" makes sense if there's lock in and you can monetize those "now locked-in users" later - but that doesn't really seem true on the consumer side. It seems pretty triv…
This is where open source models are important. The latest deepseek v4 pro model is 2-5x cheaper than Claude Sonnet 4.6. Cursor's Compose 2.5 that was just recently released is 6x cheaper than Sonnet. The state of the art models are going to get better and more expensive and smaller models are going to get cheaper. There will be a point where the intelligence of both the cheap and state of the art models are indistin…
I know it comes off as pedantic to point this out but: Those are open weight models not open source models. Closed weight models are the equivalent of SaaS. Open weight models are the equivalent of binary driver blobs or Windows software. We don't really have actual open source LLMs, which would need to publicly release their training data and technique so you could train a similar model yourself, or use their work a…
This is where open source models are important. The latest deepseek v4 pro model is 2-5x cheaper than Claude Sonnet 4.6. Cursor's Compose 2.5 that was just recently released is 6x cheaper than Sonnet. The state of the art models are going to get better and more expensive and smaller models are going to get cheaper. There will be a point where the intelligence of both the cheap and state of the art models are indistin…
> The state of the art models are going to get better and more expensive and smaller models are going to get cheaper. Why do you think this will be true? Right now I see the major US labs betting on gaining an advantage from having way more compute, and I see Chinese labs competing with one another in a resource-scarce environment, so they place much more emphasis on compute-efficiency. But the supply chains that fee…
It will be important for Apple to have good enough, cheap local LLM models that run on-device.
If the barrier to performance shifts from fundamental model capability to context collection and management I would expect to see folks focused on that problem continuing to drive open-weight LLM model development in some shape or form.
EDIT: [ IGNORE THIS COMMENT -- IT IS WRONG - I had a "bad math moment" myself ] The math seems off. How is 7.8 million vs 4 million 95% more expensive. Article makes good points but I doubt the numbers as they don’t add up. Still agree with the conclusion though.
This is where open source models are important. The latest deepseek v4 pro model is 2-5x cheaper than Claude Sonnet 4.6. Cursor's Compose 2.5 that was just recently released is 6x cheaper than Sonnet. The state of the art models are going to get better and more expensive and smaller models are going to get cheaper. There will be a point where the intelligence of both the cheap and state of the art models are indistin…
I know it comes off as pedantic to point this out but: Those are open weight models not open source models. Closed weight models are the equivalent of SaaS. Open weight models are the equivalent of binary driver blobs or Windows software. We don't really have actual open source LLMs, which would need to publicly release their training data and technique so you could train a similar model yourself, or use their work a…
Your comment is wrong both theoretically and practically.
First, the theory. The idea that model weights are "binary driver blobs" is technically wrong. I don't know why this is so common on a technical site, but anyway. An LLM model consists of 3 main parts: The architecture, the inference code, and some values. All of these, combined, make an LLM.
Another important aspect, that is widely misunderstood and will become apparent later is that a model is created by deciding the architecture, and then initialised with some values. Those values can be all 0s, all 1s, or random. (in practice it's random but that's irrelevant). Technically, once a model is initialised, that's it. That is a model. If released, that would be, even for the most pedantic absolutists, undoubtably open source.
Then, that model is being adapted. The most important thing to understand here, is that this is the preferred way of modifying a model. Actually, the only way. You can't (yet) come later and decide to change something in the architecture. Youc an only change the values. That process is called training (pre, mid, post, etc). The process itself is the same for the model creators, as it is for you. The technical process. The means, know-how, etc. is different.
Now, what licensing does, and the only thing that licensing can do is to give you rights to inspect, modify and release that model. That's it. A license will never give you (it cannot) the right to have the internal IP, knowledge, know-how or the "why's" on how the model was edited. That's on you. You have the right to modify, but you can't get the right to know how others have modified it, from a license file. Never had, never will.
(a simplified version of this is to think about an algorithm to control a drone. Usually that'd be a pid controller. Imagine someone releases under an open source license, an algorithm. That algorithm consists of architecture, loop code, and some values. Even if those values are all set to 0.5 (in which case your drone might crash) or any other values, the values themselves do not change the status of the code. It's still open source, even if the values are fixed, or random, or dreampt up by the original coder, or received from the aliens themselves)
I mentioned above that editing the values of a model is the preffered way of modifying the model, and that's exactly what Apache 2.0 defines as "source code".
> "Source" form shall mean the preferred form for making modifications, including but not limited to software source code, documentation source, and configuration files.
----
Now, the practice. In practice, we do have fully open (open data, open training code, open source models) models. Apertus, from Switzerland and Olmo from the US. Don't get me wrong, it's absolutely great that we have these models, they are very important for the community, and they do help inform everyone about what works, what doesn't, and so on. But ... no-one uses them. Because they are not at the top, compared to other models.
And, on a technical note, the idea that "dataset" + training code = bit-for-bit recreation is also not true. Anyone that has done any large scale training can tell you that. Between the randomness inherent in the process, the occasional training run re-starts and so on, you will never get the same model twice (at reasonable scales), even if you'd have the available compute. Which, let's be serious, no-one at home has. So... yeah. It's a pointless aspect to care for anyway.
Earlier quoted context omitted.
The currently-released models don't really go away. Even if they collectively only release a new model every few years for the sake of influence and public image, that's plenty enough to keep the competitive aspect going.
>Even if they collectively only release a new model every few years for the sake of influence and public image, that's plenty enough to keep the competitive aspect going. This is unpersuasive. Why would AI companies (American or Chinese) stop subsidizing tokens, but keep doing open model releases? At least for the former you can argue it's a lead generation tool for enterprise contracts (eg. hobbyist uses claude code…
M2.7 is 230B and was designed to run inference on two (2!) shity Ascend GPUs (Huawei's first GPU manufactured in China). That's why they can offer a plan at 1/2 of the price of Antropic and probably still make a revenue.
Earlier quoted context omitted.
From what I've seen pretty much every company is limited by hardware supply, to the level where's there complaints from current customers about the speed of new customer growth is exceeding their ability to service them properly. And "growth at all costs" makes sense if there's lock in and you can monetize those "now locked-in users" later - but that doesn't really seem true on the consumer side. It seems pretty triv…
Your concern about their business is that demand for their products is growing so stratospherically that they cannot meet that demand easily? I mean that's like an A+ scorecard for any business. Everyone in business would dream of such a scenario. That's called a luxury problem.
I think the idea of "all growth is good no matter the cost" has been taken to an extreme.
The pricing so far has been a classic case of loss-leader to build market share and ramp up until you can find a moat. Normally, the huge cost of training would provide such a moat, or the amount of training data required, but both of those seem to have been overcome by enough players to keep the ball in play. The next target to keep out the riffraff seems to be "Gigawatts of Data Center" (gack, I hate that metric!) and you might think that it would hold, given the finite size of the planet.
But in space, no one can hear you bleed cash, so...