- not if apple m6 mac studio comes with 1TB of RAM and 144 core cpu / gpu
No, local models will not win
11–20 of 23 posts
Re: No, local models will not win
#12> For the setup price alone of a low-end home lab1, you could buy several years of a paid subscription to one of the AI providers. The power costs would come out to around $50-$300 per month, depending on how much inference you’re running: again, the price of a couple more paid subscriptions. Okay, but I already have multiple computers capable of running local models with acceptable performance, so for me the cost is…
Let’s put numbers on it.
The mean price of electricity for me is about 40c per kWh: about as high as anywhere in the US.
There are about 730 hours in a month, and a consumer machine can sustain maybe 500W, so we can spend 0.4 * 0.5 * 730 = 146 dollars on compute power per box unless we start buying special stuff.
Re: No, local models will not win
#13> For the setup price alone of a low-end home lab1, you could buy several years of a paid subscription to one of the AI providers. The power costs would come out to around $50-$300 per month, depending on how much inference you’re running: again, the price of a couple more paid subscriptions. Okay, but I already have multiple computers capable of running local models with acceptable performance, so for me the cost is…
Let’s put numbers on it. The mean price of electricity for me is about 40c per kWh: about as high as anywhere in the US. There are about 730 hours in a month, and a consumer machine can sustain maybe 500W, so we can spend 0.4 * 0.5 * 730 = 146 dollars on compute power per box unless we start buying special stuff.
For my area (nw Iowa) where electricity is about 10¢ per kWh, that'd be like $43.8 ish? In reality much less for me personally with a Mac Studio, since it doesn't go anywhere near 500W
Re: No, local models will not win
#14There is one more possibility. Mainboards all get shared memory and memory costs go down below $1/GiB. That makes GPUs cheap, as they do not come with memory. Then local models become the norm for many, because you can buy a 10 TiB machine for 10k, and upgrade the GPU when needed. I guess that models will not grow endlessly, ones the growth in size flattens, the demand for more and more memory flattens. So, IMHO, long term local models will win. However, short to medium term it may not be. My 2 cent.
Re: No, local models will not win
#15[dead]
Re: No, local models will not win
#16If they dont the gap between rich and poor will be unsustainable.
[dead]
Re: No, local models will not win
#17If they dont the gap between rich and poor will be unsustainable.
Isn’t it already?
Re: No, local models will not win
#18They only need to be good enough to solve specific problems, that is already a victory on my book.
Re: No, local models will not win
#19> No, local models will not win Win what? Money, fame? That's not the point of small models, freedom is the point. What's this occultist obsession with "There shall be only one" monopolies? Why only one? That's so irrational and childish.
Win as the dominant for of LLM interaction. Every so often someone hypes local models as "Don't pay for Claude check out this local modal" and the local modal isn't at all equivalent. That doesn't mean that local models are going anytwhere (as the article motes they're useful) but if you want the best we're stuck with expensive remote models right now.
The fallacy is that being the best in everything isn't that relevant for most folks, in many scenarios being good enough is sufficient.
Re: No, local models will not win
#20"In a year you might be able to run something about as strong as GPT-5.6-Sol on your laptop. But by then, you’ll think of GPT-5.6-Sol as too weak to be useful." This assumption is unfounded and invalidates almost the entire premise of this piece