Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Local AI is driving the biggest change in laptops in decades

spectrum.ieee.org

Post reply on HN
Print view View original

241–250 of 273 posts

Re: Local AI is driving the biggest change in laptops in decades

#241

Post by jeremyjh »

nl wrote: post #221
jeremyjh wrote: post #200

Earlier quoted context omitted.

Datacenters full of GPU hosts aren't like dark fiber - they require massive ongoing expense, so the unit economics have to work really well. It is entirely possible that some overbuilt capacity will be left idle until it is obsolete.

The ongoing costs are mostly power, and aren't that massive compared to the investment. No one is leaving an H100 cluster not running because the power costs too much - this is why remnants markets like Vast.ai exist.

They absolutely will leave them idle if the market is so saturated that no one will pay enough for tokens to cover power and other operational costs. Demand is elastic but will not stretch forever. The build out assumes new applications with ROI will be found, and I'm sure they will be, but those will just drive more investment. A massive over build is inevitable.

Re: Local AI is driving the biggest change in laptops in decades

#242

Post by dangus »

DANmode wrote: post #206
dangus wrote: post #204

Earlier quoted context omitted.

Global Mac marketshare is actually higher than the US: https://www.mactech.com/2025/03/18/the-mac-now-has-14-8-of-t...

Less than 1 in 5 doesn’t feel like huge market share, but it’s more than I have!

Apple outsells Lenovo, if that puts it in a different perspective.

Re: Local AI is driving the biggest change in laptops in decades

#243

Post by dangus »

layer8 wrote: post #240
dangus wrote: post #203

Earlier quoted context omitted.

It's actually closer to 20% globally. Apple now outsells Lenovo: https://www.mactech.com/2025/03/18/the-mac-now-has-14-8-of-t...

I meant market share in terms of installed base: https://gs.statcounter.com/os-market-share/desktop/worldwide...

macOS and OS X are split on this graph, and “Unknown” could be anything? This might actually show Apple install base close to 20%.

Re: Local AI is driving the biggest change in laptops in decades

#244

Post by FuckButtons »

adrianN wrote: post #202
zmmmmm wrote: post #185

Earlier quoted context omitted.

if you calculate depreciation and running costs on a new car in most places - I think it probably would be.

If Uber were cheaper than the depreciation and running costs of a car, what would be left for the driver (and Uber)?

If you’re using uber to and from work, presumably you would buy a car that’s worth more than the 10 year old Prius your uber driver has 200k miles on.

Re: Local AI is driving the biggest change in laptops in decades

#245

Post by aurareturn »

fnord77 wrote: post #215
gcanyon wrote: post #201

Earlier quoted context omitted.

If Apple offered a reasonably-priced laptop with more than 24gb of memory (I'm writing this on a maxed-out Air) I'd agree. I've been buying Apple laptops for a long time, and buying the maximum memory every time. I just checked, and I see that now you can get 32gb. But to get 64gb I think you have to spend $3700 for the MBMax, and 128gb starts at $4500, almost 3x the 32gb Air's price. And as far as I understand it, a…

it's astonishing how apple gouges on the memory and ssd upgrade prices (I'm on an M1 w/ 64Gb/4Tb). That said they have some elasticity when it comes to the DRAM shortage.

They gouge you on RAM and SSD but provide a far better overall machine for the price than Windows laptops.

Re: Local AI is driving the biggest change in laptops in decades

#246

Post by nl »

jeremyjh wrote: post #241
nl wrote: post #221

Earlier quoted context omitted.

The ongoing costs are mostly power, and aren't that massive compared to the investment. No one is leaving an H100 cluster not running because the power costs too much - this is why remnants markets like Vast.ai exist.

They absolutely will leave them idle if the market is so saturated that no one will pay enough for tokens to cover power and other operational costs. Demand is elastic but will not stretch forever. The build out assumes new applications with ROI will be found, and I'm sure they will be, but those will just drive more investment. A massive over build is inevitable.

Of course!

But the operational costs are much lower than some people in this thread seem to think.

You can find a safe margin for the price by looking at aggregators.

https://gpus.io/gpus/h100 is showing $1.83/hour lowest price, around $2.85 average.

That easily pays running costs - a H100 server with cooling etc is around $0.10/hour to keep running

And a massive overbuild pushes prices down not up!

Re: Local AI is driving the biggest change in laptops in decades

#247

Post by seanmcdirmid »

KellyCriterion wrote: post #234
seanmcdirmid wrote: post #179

Earlier quoted context omitted.

Running an LLM locally means you never have to worry about how many tokens you've used, and also it allows for a lot of low latency interactions on smaller models that can run quickly. I don't see why consumer hardware won't evolve to run more LLMs locally. It is a nice goal to strive for, which consumer hardware makers have been missing for a decade now. It is definitely achievable, especially if you just care about…

isnt this what all these NPUs are created for?

I haven’t seen an NPU that can compete with a GPU yet. Maybe for really small models, I’m still not sure where they are going with those.

Re: Local AI is driving the biggest change in laptops in decades

#248

Post by almosthere »

kylec wrote: post #171
tossandthrow wrote: post #136

Earlier quoted context omitted.

Try running anything interesting on these 8gb of ram. You need 96gb or 128gb to do non trivial things. That is not yet 749 usd

The M4 starts with 16GB, though that can also be tight for local LLMs. You can get one with 24GB for $1149 right now though, which is good value.

899 at B&H started today 12/24

Re: Local AI is driving the biggest change in laptops in decades

#249

Post by dehugger »

mft_ wrote: post #228
gcanyon wrote: post #201

Earlier quoted context omitted.

If Apple offered a reasonably-priced laptop with more than 24gb of memory (I'm writing this on a maxed-out Air) I'd agree. I've been buying Apple laptops for a long time, and buying the maximum memory every time. I just checked, and I see that now you can get 32gb. But to get 64gb I think you have to spend $3700 for the MBMax, and 128gb starts at $4500, almost 3x the 32gb Air's price. And as far as I understand it, a…

You’re not wrong that Apple’s memory prices are unpleasant, but also consider the competition - in this context (running LLMs locally) laptops with large amounts of fast memory that can be purposed for the GPU . This limits you to Apple or one specific AMD processor at present. An HP Zbook with an AMD 395+ and 128Gb of memory apparently lists for $4049 [0] An ASUS ROG Flow z13 with the same spec sells for $2799 [1] -…

The framework desktop will get you the 395+ and 128gb of ram for 2k USD.

Re: Local AI is driving the biggest change in laptops in decades

#250

Post by mirror_neuron »

jki275 wrote: post #160
azuanrb wrote: post #119

Earlier quoted context omitted.

You still need ridiculously high spec hardware, and at Apple’s prices, that isn’t cheap. Even if you can afford it (most won't), the local models you can run are still limited and they still underperform. It’s much cheaper to pay for a cloud solution and get significantly better result. In my opinion, the article is right. We need a better way to run LLMs locally.

I bought my M1 Max w/ 64gb of ram used. It's not that expensive. Yes, the models it can run do not perform like chatgpt or claude 4.5, but they're still very useful.

I’m curious to hear more about how you get useful performance out of your local setup. How would you characterize the difference in “intelligence” of local models on your hardware vs. something like chatgpt? I imagine speed is also a factor. Curious to hear about your experiences in as much detail as you’re willing to share!
Post reply on HN