As inflation plays with 10-year highs, fuel prices go up permanently (thanks to the end of middle east oil), and NIMBYs chase datacenters out of their regions, I think it's inevitable that AI is going to go up in price. It's just a question of how much. Companies should have a fallback plan to either switch AI providers, or replace AI with a pool of new hires quickly.
AI subscriptions are a ticking time bomb for enterprise
181–190 of 426 posts
Re: AI subscriptions are a ticking time bomb for enterprise
#182Earlier quoted context omitted.
The economics of local AI just doesn’t make sense. A model like Opus is - supposedly - something like 5T parameters, which is likely something like 3TB of GPU memory. Local models never reach the % utilization that cloud providers have (80%+), and they’re always going to be much better than local models for this reason.
Running local applications is less efficient than thin clients to the cloud generally, not just in LLMs. The trick is that you can get to the point where it's effective enough, and affordable enough, that the control and availability factors become dominant.
My point is that you will always get much more value / $ by using cloud based solutions.
Re: AI subscriptions are a ticking time bomb for enterprise
#183Earlier quoted context omitted.
> within a few years we will be running local models as good as today’s frontier models I seriously doubt it. Scaling is already strained (don't buy into the "exponential" hype). And, in any case, the competition will be against the frontier models that will exist in two years.
> I seriously doubt it. Scaling is already strained (don't buy into the "exponential" hype). And, in any case, the competition will be against the frontier models that will exist in two years. The big question I'd be asking if I was investing in one of the big players is if those changes are "it can do 99% instead of 97% of the tasks a user will throw at it" (at which point going local and taking back cost control/ow…
Exactly the right argument. Local LLM doesn’t need to outrun the bear (outperform data centers) it only needs to outrun its friend (total cost of ownership).
Re: AI subscriptions are a ticking time bomb for enterprise
#184Earlier quoted context omitted.
What's the cheapest PC you can buy today that will comfortably run Gemma 4 and everything else you want it to run at the same time? And how many tokens would that buy?
I run it on my 4 year old MBP and get 10 tok/s. With the RAM shortage buying anything new today is a nightmare but anyone with a reasonably modern Mac could run it at q6 probably. It is mostly a toy as 4o models weren’t really suitable for real work IMO but at least it won’t ever give me a refusal.
At 10toks, are you using it interactively or do you submit a prompt and come back to it later? I always thought it would make sense to just do conversations over email, asynchronously, the model can take all the time it needs and get back to me when it has an answer.
Re: AI subscriptions are a ticking time bomb for enterprise
#185[flagged]
Re: AI subscriptions are a ticking time bomb for enterprise
#186I had a conversation with Claude yesterday about this very topic. The AI was pretty candid about the issue and said many of the same things the author said. Now I am not sure if I went in with an unintended bias and it just went into full sycophant mode, I tried to be neutral in my prompts, along the lines of the implications of integrating AI into processes when the true cost is not being charged. But it was obvious that even moderate usage is a loss leader, so heavy users with agentic workloads are in a risky situation and should think long and hard about their business model if costs slowly trickle up in the triple, quadruple etc etc range.
I will continue to use it as an assistant that does the menial stuff quicker than I ever could, but it's just too early to let it do stuff that would hurt if it disappeared. Enjoy it while it lasts.
Re: AI subscriptions are a ticking time bomb for enterprise
#187Replacing your workers with AI: --You lose control over their "salary" --You lose control over their "schedule" --Your company becomes reliant on another party that does not share your interests or values, and can stop working for you on a whim for any reason But AI is definitely good and trade unions are definitely bad, apparently...
If only there was a way to think beyond direct substitution.
Re: AI subscriptions are a ticking time bomb for enterprise
#188I had a conversation with Claude yesterday about this very topic. The AI was pretty candid about the issue and said many of the same things the author said. Now I am not sure if I went in with an unintended bias and it just went into full sycophant mode, I tried to be neutral in my prompts, along the lines of the implications of integrating AI into processes when the true cost is not being charged. But it was obvious…
I think a solution could be local hardware acceleration the diffecult thing to achieve is not leaking dmodel data, since yeah that is obviously a nogo for antropic, openai, etc
Re: AI subscriptions are a ticking time bomb for enterprise
#189The entire problem with "AI" is that it's easy to do without. The AI companies know it, the users know it - even the most pro AI agent manager knows it. Thought experiment: remove AI from the world right now, all of it - what do you have? Business as usual. This article doesn't do enough to underscore that - dreaded be the day I need to get an actual engineer to review a PR, right?
It's always weird when people are suggesting to use some AI tool for the most mundane and generic kind of task. Like it's some kind of pet that will die if it's not used every once in a while.
Re: AI subscriptions are a ticking time bomb for enterprise
#190Every AI subscription is a ticking time bomb for the frontier provider; within a few years we will be running local models as good as today’s frontier models with almost no cost burden. The floor will fall out of the enterprise market for all the frontier companies.
> within a few years we will be running local models as good as today’s frontier models with almost no cost burden Based on what? The RAM requirements alone are extraordinary. No, running large models on shared, dedicated hosted hardware at full utilization is going to be vastly more cost-efficient for the foreseeable future.
You can now buy 128 GB unified memory computers from AMD as commodity.
They’re still pricey, the world is still scaling up memory production, and a lot of code isn’t yet built for AMD, but we went from the Wright’s brothers first airplane to jet engines in 27 years.
I’m not sure “it’s only a few years away” but we are sure moving there fast.