Instead of programming something yourself now you'll be forced to program through the interface of someone that doesn't speak English putting what you say into an LLM. We've peaked.
Outsourcing plus local AI will soon become more economical vs. frontier labs
381–390 of 408 posts
Re: Outsourcing plus local AI will soon become more economical vs. frontier labs
#382I keep seeing this narrative involving Deepseek as an example of OSS LLMs but they are subsidizing a huge amount of tokens at cost and one can easily understand why they are doing it if one is not lazy and think critically. It's still far too costly and not effective to use Local AI that can match what the frontier models can offer, especially when the inference hardware is being heavily restricted due to geopolitica…
Do you have a source for your first claim? My impression is that deepseek designed v4 specifically for cheap inference and they are not loosing money even at 75% lower price.
Just like they had a new GPU-light training method for v3 that turned out to be more compute intensive than originally advertised?
Re: Outsourcing plus local AI will soon become more economical vs. frontier labs
#383When discussing LLM pricing, people are missing the plot. The subscription token price is 10x-40x cheaper than API pricing. Your 90$ Claude subscriptions give you close to $1000 to $4000 in equivalent API token pricing. The second issue is that the quality of the model “operator” makes a massive difference in the outcomes. Highly skilled senior devs who know how to prompt and have high agency will outperform team peo…
I learned today that the Anthropic "Enterprise" plan - the one big companies use because they need governance features and audit logs and all of that jazz - is billed at API token rates (plus $20/seat/month). So large companies are getting billed a lot more than those discount subscription plans.
Do we know that they all pay the API rates or will they negotiate individually?
Re: Outsourcing plus local AI will soon become more economical vs. frontier labs
#384When discussing LLM pricing, people are missing the plot. The subscription token price is 10x-40x cheaper than API pricing. Your 90$ Claude subscriptions give you close to $1000 to $4000 in equivalent API token pricing. The second issue is that the quality of the model “operator” makes a massive difference in the outcomes. Highly skilled senior devs who know how to prompt and have high agency will outperform team peo…
So when you need an LLM in your backend, scrape Claude Code instead of using the API :)
Re: Outsourcing plus local AI will soon become more economical vs. frontier labs
#385Earlier quoted context omitted.
There we go back to the original question: are subscriptions profitable, API pricing wildly profitable, and they just lose all that money on fixed costs like model training; or do they actually barely make money on inference? That's why talking about the profitability of inference without accounting for model training is interesting, because that is the deciding factor in whether more customers would help getting the…
Without actual data I don't know. My gut feeling is that they overall lose money on subscriptions (and especially the free tier that accounts for 95% of all users). And make thin profit (~5%) on API pricing. But it's just that. A gut feeling.
I don’t think it’s a gut feeling. It seems to be the consensus that subscriptions are still heavily subsidized
Re: Outsourcing plus local AI will soon become more economical vs. frontier labs
#386[dead]
Re: Outsourcing plus local AI will soon become more economical vs. frontier labs
#387Earlier quoted context omitted.
…the best model for agentic coding, is the top goal right now.
Good example. AFAIK they are still focusing on doing this via larger models which is a bad call. They are also focusing to hard on fully-agentic coding which, while useful to a limited extent, is not the best way to use AI for most non-trivial coding tasks.
Yeah, fully agentic coding just takes too much knowledge out of the engineer. It’s a lazy man’s button to do work and yield average results. Fun for vibe coding and not reliable enough for enterprise where quality matters. Maybe their focus will pay off in reliability, maybe not.
Re: Outsourcing plus local AI will soon become more economical vs. frontier labs
#388[flagged]
Re: Outsourcing plus local AI will soon become more economical vs. frontier labs
#389Earlier quoted context omitted.
Burning money is never sustainable. All you're actually saying is nobody can predict how long this particular bonfire will burn.
Again this is nonsense for the reasons I've already given. The costs aren't fixed.
Whether sustainability is achieved by raising prices or hoping that costs can be brought down, you have to acknowledge that the status quo is unsustainable. If it were sustainable nether change would be necessary.
Re: Outsourcing plus local AI will soon become more economical vs. frontier labs
#390Earlier quoted context omitted.
The marginal cost of AI is not 0. That's one of the big differences between this and older SaaS software. Inference costs a lot of money. Even if you're looking at just capital depreciation, it's quite expensive. I suppose it's more accurate to say marginal cost is stepwise - adding 1 new user is 0 cost if and only if your existing inference hardware covers that user's usage. As soon as you need a new server, adding…
It's not literally 0 but for most casual users it is. Programmers are heavier users and the ones that have 10 agents going wild are even more so. In that case the cost is above 0 but economies of scale means the effect is the same.
The marginal cost for _users_ (on a token plan) is $0 but not for the AI company supplying that token plan.