Live data from Hacker News

Dell admits consumers don't care about AI PCs

pcgamer.com

301–310 of 418 posts

Re: Dell admits consumers don't care about AI PCs

#301

I don't know how many others here have a CoPilot+ PC but the NPU on it is basically useless. There isn't any meaningful feature I get by having that NPU. They are far too limited to ever do any meaningful local LLM inference, image processing or generation. It handles stuff like video chat background blurring, but users' PC's have been doing that for years now without an NPU.

NPUs were pushed by Microsoft, who saw the writing on the wall: AI like chatgpt will dominate the user's experience, edge computing is a huge advantage in that regard, and Apple's hardware can do it. NPUs are basically Microsoft trying to fudge their way to a llamacpp-on-Apple-Silicon experience. Obviously it failed, but they couldn't not try.

> NPUs were pushed by Microsoft, who saw the writing on the wall: AI like chatgpt will dominate the user's experience, edge computing is a huge advantage in that regard

Then where is a demo application from Microsoft of a model that I can run locally where my user experience is so much better (faster?) if my computer has an NPU?

Re: Dell admits consumers don't care about AI PCs

#302

Earlier quoted context omitted.

Yeah, that's because the original npus were a rush job, the amd AI Max is the only one that's worth anything in my opinion.

Is that because of the actual processing unit or because they doubled the width of the memory bus?

It's because it comes with a decent iGPU, not because of the NPU inside of that. The NPU portion is still the standard tiny 50 TOPS and could be filled with normal RAM bandwidth like on a much cheaper machine.

On the RAM bandwidth side it depends if you want to look at it as "glass is half full" or "glass is half empty". For "glass is half full" the GPU has access to a ton of RAM at ~2x-4x the bandwidth of normal system memory an iGPU would have and so you can load really big models. For "glass is half empty" that GPU memory bandwidth is still nearly 2x less than a even a 5060 dGPU (which doesn't have to share any of that bandwidth with the rest of the system), but you won't fit as large of a model on a dGPU and it won't be as power efficient.

Speaking of power efficiency - it is decently power efficient... but I wouldn't run AI on battery on mine unless I was plugged in anyways as it still eats through the battery pretty quick when doing so. Great general workstation laptop for the size and wattage though.

Re: Dell admits consumers don't care about AI PCs

#303

I don't know how many others here have a CoPilot+ PC but the NPU on it is basically useless. There isn't any meaningful feature I get by having that NPU. They are far too limited to ever do any meaningful local LLM inference, image processing or generation. It handles stuff like video chat background blurring, but users' PC's have been doing that for years now without an NPU.

If you do use video chat background blurring, the NPU is more efficient at it vs using your cpu or gpu. So the feature it supports is longer battery life and less resource usage on your main chips.

Re: Dell admits consumers don't care about AI PCs

#305

I don't know how many others here have a CoPilot+ PC but the NPU on it is basically useless. There isn't any meaningful feature I get by having that NPU. They are far too limited to ever do any meaningful local LLM inference, image processing or generation. It handles stuff like video chat background blurring, but users' PC's have been doing that for years now without an NPU.

If you do use video chat background blurring, the NPU is more efficient at it vs using your cpu or gpu. So the feature it supports is longer battery life, and less resource usage on your main chips, and better performance for the things that NPUs can do. E.g higher video quality on your blurred background.

Re: Dell admits consumers don't care about AI PCs

#306

Earlier quoted context omitted.

I'd be surprised if Microsoft would sell them Windows licenses or would work with them on drivers if they don't put the Copilot key on the keyboard.

What is with Microsoft and demanding a key for every new thing they come up with?

When had Microsoft done this before? For Cortana maybe? I can't recall them ever mandating dedicated physical keys for anything other than the Windows key, but that was over 30 years ago and I assume that's not what you're talking about.

Re: Dell admits consumers don't care about AI PCs

#307

I don't know how many others here have a CoPilot+ PC but the NPU on it is basically useless. There isn't any meaningful feature I get by having that NPU. They are far too limited to ever do any meaningful local LLM inference, image processing or generation. It handles stuff like video chat background blurring, but users' PC's have been doing that for years now without an NPU.

If you do use video chat background blurring, the NPU is more efficient at it vs using your cpu or gpu. So the feature it supports is longer battery life, and less resource usage on your main chips, and better performance for the things that NPUs can do. E.g higher video quality on your blurred background.

Really, the best we can do with the NPU is a less battery intensive blurred background? R&D money well spent I guess...

Re: Dell admits consumers don't care about AI PCs

#308
post #232

Earlier quoted context omitted.

Question - from the perspective of the actual silicon, are these NPUs just another form of SIMD? If so, that's laughable sleight of hand and the circuits will be relegated to some mothball footnote in the same manner as AVX512, etc. To be fair, SIMD made a massive difference for early multimedia PCs for things like music playback, gaming, and composited UIs.

NPUs are a separate accelerator block, not in-CPU SIMD. The latter exists for matrix compute, but only in the latest version of AVX which has yet to reach consumer CPUs.

> The latter exists for matrix compute, but only in the latest version of AVX which has yet to reach consumer CPUs.

As far as I am aware, AMD implemented has implemented many parts of AVX-512 in their consumer CPUs since Zen 4:

https://en.wikipedia.org/w/index.php?title=AVX-512&oldid=133...

On the other hand, Intel still does not support AVX-512 in Raptor Lake, Meteor Lake and Arrow Lake:

> https://en.wikipedia.org/wiki/Raptor_Lake

> https://en.wikipedia.org/wiki/Meteor_Lake

> https://en.wikipedia.org/wiki/Arrow_Lake_(microprocessor)

Re: Dell admits consumers don't care about AI PCs

#309

Earlier quoted context omitted.

NPUs are still useful for LLM pre-processing and other compute-bound tasks. They will waste memory bandwidth during LLM generation phase (even in the best-case scenario where they aren't physically bottlenecked on bandwidth to begin with, compared to the iGPU) since they generally have to read padded/dequantized data from main memory as they compute directly on that, as opposed to being able to unpack it in local reg…

The idea that tokenization is what they're for is absurd - you're talking a tenth of a thousandth of a millionth of a percent of efficiency gain in real world usage, if that, and only if someone bothers to implement it in software that actually gets used. NPUs are racing stripes, nothing more. No killer features or utility, they probably just had stock and a good deal they could market and tap into the AI wave with.

NPUs aren't meant for LLMs. There are a lot more neural net tech out there than LLMs.

Re: Dell admits consumers don't care about AI PCs

#310

Earlier quoted context omitted.

I've already forgotten most assembly languages I ever used. I look forward to forgetting C++.

Last part is very common, but what's wrong with assembly languages? But I wasn't talking about forgetting one language or another, i was talking about forgetting to program completely.

Nothing at all wrong with assembly languages. I just don't need them anymore.
Post reply on HN