281–290 of 426 posts
Earlier quoted context omitted.
What they charge people says nothing about what it costs them. Off the top of my head, one confounding factor is trying to win back marketshare from Anthropic. We will only know the actually situation once Anthropic goes public and we can look at their books.
I think it's pretty safe to assume they are not losing money on inference.
[flagged]
There's plenty of sand on the planet and clever people (and AI) figuring out how to do more work with less sand and power, so any argument that AI is going to cost so much that it won't be usable, seems just preposterous.
Earlier quoted context omitted.
I think there will always be a free tier that they'll be willing to use. Even if it sounds hackneyed, those folks will still use it because many people are not discerning readers anyway.
Despite what I just said, I do hope so, because I'm really not inclined to pay for it, at least not very much. I don't need another $100-200/mo bill in my life, and it doesn't provide that level of value as a chatbot. Google is enough. I'm not sure that free tier will necessarily continue forever though, unless there is a way to monetize it (presumably by advertising, or by selling data they've gleaned about the user…
Earlier quoted context omitted.
I haven't been following anyone baking models into ASICs, is it not still necessary to pack just as many transistors onto a chip, whether it's an NPU or GPU, ASIC or not you still need to hold hundreds of gigabytes in memory, so how is it cheaper to bake it onto custom silicon than running it on commodity VRAM? (Asking because I don't know!)
Not my area either! But my understanding is that there are more efficient methods of representing static numbers when you can skip the vram lookup. https://taalas.com/ Is an example startup in this area claiming 16k tok/s on an asic for llama 8b. Qwen has a 27b model at opus 4.5 quality.
Every AI subscription is a ticking time bomb for the frontier provider; within a few years we will be running local models as good as today’s frontier models with almost no cost burden. The floor will fall out of the enterprise market for all the frontier companies.
> within a few years we will be running local models as good as today’s frontier models with almost no cost burden Based on what? The RAM requirements alone are extraordinary. No, running large models on shared, dedicated hosted hardware at full utilization is going to be vastly more cost-efficient for the foreseeable future.
Earlier quoted context omitted.
I think hardware prices will come back down once we start seeing more efficiency improvements in models and hardware, and once more people and companies self-host models (which seems to be happening more and more these days). I think the massive infra/hardware expenditures of OpenAI and the like are going to end up unnecessary, leading to hardware price drops.
If companies decide to self-host, wouldn't that drive the demand and therefore prices up? Most companies currently do not have the needed infrastructure.
Earlier quoted context omitted.
"It costs OpenAI less money to serve GPT-5.5 than GPT-4." does it though? do you have the numbers? Or you just making stuff up?
We used to not know, but now because open source models are being hosted and served by people whose only incentive is making profit on directly running inference, we have a ballpark idea.
It would be more surprising if the surrounding architecture hasn't significantly diverged. If it _hasn't_ significantly diverged, then given the performance difference it would imply that the frontier models have significantly greater param counts, which would result in a higher cost.
[flagged]