Local AI is driving the biggest change in laptops in decades

This article is so dumb. It totally ignores the memory price explosion that will make large fast memory laptops unfeasible for years and states stuff like this:

> How many TOPS do you need to run state-of-the-art models with hundreds of millions of parameters? No one knows exactly. It’s not possible to run these models on today’s consumer hardware, so real-world tests just can’t be done.

We know exactly the performance needed for a given responsiveness. TOPS is just a measurement independent from the type of hardware it runs on..

The less TOPS the slower the model runs so the user experience suffers. Memory bandwidth and latency plays a huge role too. And context, increase context and the LLM becomes much slower.

We don't need to wait for consumer hardware until we know much much is needed. We can calculate that for given situations.

It also pretends small models are not useful at all.

I think the massive cloud investments will put pressure away from local AI unfortunately. That trend makes local memory expensive and all those cloud billions have to be made back so all the vendors are pushing for their cloud subscriptions. I'm sure some functions will be local but the brunt of it will be cloud, sadly.

Re: Local AI is driving the biggest change in laptops in decades

#6

Post by noosphr » Tue, Dec 23, 2025, 3:20 AM UTC

wkat4242 wrote: post #4

aappleby wrote: post #3

I predict we will see compute-in-flash before we see cheap laptops with 128+ gigs of ram.

Yeah especially since what is happening in the memory market

Feast and famine.

In three years we will be swimming in more ram than we know what to do with.

Re: Local AI is driving the biggest change in laptops in decades

#7

Post by esses » Tue, Dec 23, 2025, 3:23 AM UTC

I spent a good 30 seconds trying to figure out what DDS was an acronym for in this context.

Re: Local AI is driving the biggest change in laptops in decades

#8

Post by seanmcdirmid » Tue, Dec 23, 2025, 3:25 AM UTC

I’ve been running LLMs on my laptop (M3 Max 64GB) for a year now and I think they are ready, especially with how good mid sized models are getting. I’m pretty sure unified memory and energy efficient GPUs will be more than just a thing on Apple laptops in the next few years.

Re: Local AI is driving the biggest change in laptops in decades

#9

Post by Morromist » Tue, Dec 23, 2025, 3:25 AM UTC

I was in the market for a laptop this month. Many new laptops now advertise AI features like this "HP OmniBook 5 Next Gen AI PC" which advertises:

"SNAPDRAGON X PLUS PROCESSOR - Achieve more everyday with responsive performance for seamless multitasking with AI tools that enhance productivity and connectivity while providing long battery life"

I don't want this garbage on my laptop, especially when its running of its battery! Running AI on your laptop is like playing Starcraft Remastered on the Xbox or Factorio on your steamdeck. I hear you can play DOOM on a pregnancy test too. Sure, you can, but its just going to be a tedious inferior experiance.

Really, this is just a fine example of how overhyped AI is right now.

Re: Local AI is driving the biggest change in laptops in decades

#10

Post by vegabook » Tue, Dec 23, 2025, 3:30 AM UTC

wkat4242 wrote: post #5

This article is so dumb. It totally ignores the memory price explosion that will make large fast memory laptops unfeasible for years and states stuff like this: > How many TOPS do you need to run state-of-the-art models with hundreds of millions of parameters? No one knows exactly. It’s not possible to run these models on today’s consumer hardware, so real-world tests just can’t be done. We know exactly the performan…

also, state of the art models have hundreds of _billions_ of parameters.