Local Models Will Not Win
seangoedecke.com
Local Models Will Not Win
1–6 of 6 posts
Re: Local Models Will Not Win
#2Local models are never going to be as powerful.
Yes, and PCs will never be as powerful as mainframes.
And smart phones will never be as powerful as PCs.
But which ones are more popular and used more for computing?
In other words, "power" isn't the only factor that needs to be considered.
Re: Local Models Will Not Win
#3> Local models are never going to be as powerful. I think this point should be obvious: all of the current frontier models (closed and open-weights) are far too big to run on anything but a full GPU cluster in a datacenter
Diminishing returns with respect to scale (a 10X larger model is typically not 10X better at any given task) has so far meant that, even when datacenter compute grows faster than individual compute, the gap in quality between hosted and local models has generally shrunk. There are still plenty of tasks where that extra gain in quality is noticeable, but I feel there are also an increasing number of "saturated" tasks where it really doesn't matter.
Which puts more focus on other factors. Local models are private, low-latency, work offline, and can be tinkered with to your liking - like changing the system prompt to avoid refusals. I would not trust a hosted model to classify my documents, for example.
Re: Local Models Will Not Win
#4Local models are never going to be as powerful. Yes, and PCs will never be as powerful as mainframes. And smart phones will never be as powerful as PCs. But which ones are more popular and used more for computing? In other words, "power" isn't the only factor that needs to be considered.
How useful are smartphones without connection to the mainframes that back them? While I'd love for the gigaherz of computing I hold in the palm of my hand to be useful unto itself, if it doesn't connect to Uber, Gmail, Netflix, Apple, et al, how useful those gigaherz of computing are is irrelevant, unless I want to do some local computing.
Re: Local Models Will Not Win
#5Local models are never going to be as powerful. Yes, and PCs will never be as powerful as mainframes. And smart phones will never be as powerful as PCs. But which ones are more popular and used more for computing? In other words, "power" isn't the only factor that needs to be considered.
How useful are smartphones without connection to the mainframes that back them? While I'd love for the gigaherz of computing I hold in the palm of my hand to be useful unto itself, if it doesn't connect to Uber, Gmail, Netflix, Apple, et al, how useful those gigaherz of computing are is irrelevant, unless I want to do some local computing.
How useful are mainframes without connections to smartphones?
Communication is a different issue. It is essential in today's world regardless of the "power" of the pieces being connected.
Re: Local Models Will Not Win
#6> Local models are more expensive and less efficient
> Local models are never going to be as powerful.
Proven wrong empirically by Nvidia back in November: https://arxiv.org/abs/2511.21689
"On HLE, Orchestrator achieves a score of 37.1%, outperforming GPT-5 (35.1%) while being 2.5x more efficient."
Cute blog though