Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Local AI is driving the biggest change in laptops in decades

spectrum.ieee.org

Post reply on HN
Print view View original

221–230 of 273 posts

Re: Local AI is driving the biggest change in laptops in decades

#221

Post by nl »

jeremyjh wrote: post #200
nl wrote: post #183

Earlier quoted context omitted.

100% yes. The amount of compute in the world is doubling over 2 years because of the ongoing investment in AI (!!) In some scenario where new investment stops flowing and some AI companies go bankrupt all that compute will be looking for a market. Inference providers are already profitable so with cheaper hardware it will mean even cheaper AI systems.

Datacenters full of GPU hosts aren't like dark fiber - they require massive ongoing expense, so the unit economics have to work really well. It is entirely possible that some overbuilt capacity will be left idle until it is obsolete.

The ongoing costs are mostly power, and aren't that massive compared to the investment.

No one is leaving an H100 cluster not running because the power costs too much - this is why remnants markets like Vast.ai exist.

Re: Local AI is driving the biggest change in laptops in decades

#222

Post by nl »

blibble wrote: post #207
nl wrote: post #183

Earlier quoted context omitted.

100% yes. The amount of compute in the world is doubling over 2 years because of the ongoing investment in AI (!!) In some scenario where new investment stops flowing and some AI companies go bankrupt all that compute will be looking for a market. Inference providers are already profitable so with cheaper hardware it will mean even cheaper AI systems.

> The amount of compute in the world is doubling over 2 years because of the ongoing investment in AI (!!) which is funded by the dumping when the bubble pops: these DCs are turned off and left to rot, and your capacity drops by a factor of 8192

> which is funded by the dumping

What dumping do you mean?

Are you implying NVidia is selling H200s below cost?

If not then you might be interested to see that Deepseek has released there inference costs here: https://github.com/deepseek-ai/open-infra-index/blob/main/20...

If they are losing money it's because they have a free app they are subsidizing, not because the API is underpriced.

Re: Local AI is driving the biggest change in laptops in decades

#223

Post by re-thc »

seanmcdirmid wrote: post #170
re-thc wrote: post #117

Earlier quoted context omitted.

> Why do we think it will be different now? Margins. AI usage can pay a lot more. Even if they sell less than can still be more profitable. In the past there wasn’t a high margin usage. Servers didn’t charge such a high premium.

Do you not think that some DRAM producer isn't going to see the high margins as a signal to create more capacity to get ahead of the other DRAM producers? This is how it always has worked before, but somehow it is different this time?

> Do you not think that some DRAM producer isn't going to see the high margins as a signal to create more capacity to get ahead of the other DRAM producers?

They took the bite during COVID and failed, so there's still fear from over supply.

Re: Local AI is driving the biggest change in laptops in decades

#224

Post by sipjca »

fwipsy wrote: post #54
sipjca wrote: post #24

Earlier quoted context omitted.

> It may be possible to run a "good enough" model on consumer hardware eventually 10-15 years?!!!! What is the definition of good enough? Qwen3 8B or A30B are quite capable models which run on a lot of hardware even today. SOTA is not just getting bigger, it's also getting more intelligence and running it more efficiently. There have been massive gains in intelligence at the smaller model sizes. It is just highly tas…

"Good enough" has to mean users won't be frequently frustrated if they transition to it from a frontier model. > it is highly task dependent... much could be done without that level of intelligence This is an enthusiast's glass-half-full perspective, but casual end users are gonna have a glass-half-empty perspective. Quen3-8B is impressive, but how many people use it as a daily driver? Most casual users will toss it…

How many consumers (not business) are genuinely using frontier models? You think OpenAI and Anthropic will forever serve the most intelligent models to free users? Heck they don’t already

Efficiency gains exist and likely will continue, as well as hardware generally accelerating, as software and hardware starts to become co-optimized. This will take time no doubt but 10-15 years is hilariously long in this world. The iPhone has barely been out that long

And to be clear I think the other arguments are valid I just think the timeline is out of whack

Re: Local AI is driving the biggest change in laptops in decades

#225

Post by ACCount37 »

djsjajah wrote: post #104
ACCount37 wrote: post #62

Earlier quoted context omitted.

Nah, that's just plain wrong. First, GPGPU is powerful and flexible. You can make an "AI-specific accelerator", but it wouldn't be much simpler or much more power-efficient - while being a lot less flexible. And since you need to run traditional graphics and AI workloads both in consumer hardware? It makes sense to run both on the same hardware. And bandwidth? GPUs are notorious for not being bandwidth starved. 4K@60…

GPUs might not be bandwidth starved most of the time, but they absolutely are when generating text from an llm. It’s the whole reason why low precision floating point numbers are being pushed by nvidia.

That's memory bandwidth, not I/O. Unless your LLM doesn't fit into VRAM.

Re: Local AI is driving the biggest change in laptops in decades

#226

Post by seanmcdirmid »

re-thc wrote: post #223
seanmcdirmid wrote: post #170

Earlier quoted context omitted.

Do you not think that some DRAM producer isn't going to see the high margins as a signal to create more capacity to get ahead of the other DRAM producers? This is how it always has worked before, but somehow it is different this time?

> Do you not think that some DRAM producer isn't going to see the high margins as a signal to create more capacity to get ahead of the other DRAM producers? They took the bite during COVID and failed, so there's still fear from over supply.

It only works if they collude on keeping supply steady. If anyone gets greedy for a bigger share of the AI pie, then it implodes quickly. Not all DRAM is made in South Korea so some nationalism will muddy the waters as well.

Re: Local AI is driving the biggest change in laptops in decades

#227

Post by TeMPOraL »

m4rtink wrote: post #174
whazor wrote: post #115

Earlier quoted context omitted.

But economically, it is still much better to buy a lower spec't laptop and to pay a monthly subscription for AI. However, I agree with the article that people will run big LLMs on their laptop N years down the line. Especially if hardware outgrows best-in-class LLM model requirements. If a phone could run a 512GB LLM model fast, you would want it.

Are you sure the subscription will still be affordable after the venture capital flood ends and the dumping stops?

Doesn't matter now. GP can revisit the math and buy some hardware once the subscription prices actually grow too high.

Re: Local AI is driving the biggest change in laptops in decades

#228

Post by mft_ »

gcanyon wrote: post #201
jwr wrote: post #96

The author seems unaware of how well recent Apple laptops run LLMs. This is puzzling and puts into question the validity of anything in this article.

If Apple offered a reasonably-priced laptop with more than 24gb of memory (I'm writing this on a maxed-out Air) I'd agree. I've been buying Apple laptops for a long time, and buying the maximum memory every time. I just checked, and I see that now you can get 32gb. But to get 64gb I think you have to spend $3700 for the MBMax, and 128gb starts at $4500, almost 3x the 32gb Air's price. And as far as I understand it, a…

You’re not wrong that Apple’s memory prices are unpleasant, but also consider the competition - in this context (running LLMs locally) laptops with large amounts of fast memory that can be purposed for the GPU. This limits you to Apple or one specific AMD processor at present.

An HP Zbook with an AMD 395+ and 128Gb of memory apparently lists for $4049 [0]

An ASUS ROG Flow z13 with the same spec sells for $2799 [1] - so cheaper than Apple, but still a high price for a laptop.

[0] https://hothardware.com/reviews/hp-zbook-ultra-g1a-128gb-rev...

[1] https://www.hidevolution.com/asus-rog-flow-z13-gz302ea-xs99-...

Re: Local AI is driving the biggest change in laptops in decades

#230

Post by jwr »

gcanyon wrote: post #201
jwr wrote: post #96

The author seems unaware of how well recent Apple laptops run LLMs. This is puzzling and puts into question the validity of anything in this article.

If Apple offered a reasonably-priced laptop with more than 24gb of memory (I'm writing this on a maxed-out Air) I'd agree. I've been buying Apple laptops for a long time, and buying the maximum memory every time. I just checked, and I see that now you can get 32gb. But to get 64gb I think you have to spend $3700 for the MBMax, and 128gb starts at $4500, almost 3x the 32gb Air's price. And as far as I understand it, a…

I think pricing is just one dimension of this discussion — but let's dive into it. I agree it's a lot of money. But what are you comparing this pricing to?

From what I understand, getting a non-Apple solution to the problem of running LLMs in 64GB of VRAM or more has a price tag that is at least double of what you mentioned, and likely has another digit in front if you want to get to 128GB?

Post reply on HN