Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Dell admits consumers don't care about AI PCs

pcgamer.com

Post reply on HN
Print view View original

151–160 of 418 posts

Re: Dell admits consumers don't care about AI PCs

#151

Post by GrantMoyer »

pseudosavant wrote: post #133

I don't know how many others here have a CoPilot+ PC but the NPU on it is basically useless. There isn't any meaningful feature I get by having that NPU. They are far too limited to ever do any meaningful local LLM inference, image processing or generation. It handles stuff like video chat background blurring, but users' PC's have been doing that for years now without an NPU.

The idea is that NPUs are more power efficient for convolutional neural network operations. I don't know whether they actually are more power efficent, but it'd be wrong to dismiss them just because they don't unlock new capabilties or perform well for very large models. For smaller ML applications like blurring backgrounds, object detection, or OCR, they could be beneficial for battery life.

Re: Dell admits consumers don't care about AI PCs

#152

Post by [deleted] »

pseudosavant wrote: post #133

I don't know how many others here have a CoPilot+ PC but the NPU on it is basically useless. There isn't any meaningful feature I get by having that NPU. They are far too limited to ever do any meaningful local LLM inference, image processing or generation. It handles stuff like video chat background blurring, but users' PC's have been doing that for years now without an NPU.

[deleted]

Re: Dell admits consumers don't care about AI PCs

#153

Post by rzzzt »

Galanwe wrote: post #150

I have a "Copilot" button on my new ThinkPad. I have yet to understand what it does that necessitates a dedicated button. On Linux it does nothing, on Windows it tells me I need an Office 365 plan to use it. Like... What the hell... They literally placed a paywalled Windows only physical button on my laptop. What next, an always-on screen for ads next to the trackpad?

It's equivalent to Win + Shift + F23 so you can map it to some useful action if you have a suitable utility at hand.

Re: Dell admits consumers don't care about AI PCs

#154

Post by margalabargala »

GrantMoyer wrote: post #151
pseudosavant wrote: post #133

I don't know how many others here have a CoPilot+ PC but the NPU on it is basically useless. There isn't any meaningful feature I get by having that NPU. They are far too limited to ever do any meaningful local LLM inference, image processing or generation. It handles stuff like video chat background blurring, but users' PC's have been doing that for years now without an NPU.

The idea is that NPUs are more power efficient for convolutional neural network operations. I don't know whether they actually are more power efficent, but it'd be wrong to dismiss them just because they don't unlock new capabilties or perform well for very large models. For smaller ML applications like blurring backgrounds, object detection, or OCR, they could be beneficial for battery life.

Not sure about all NPUs, but TPUs like Google's Coral accelerator are absolutely, massively more efficient per watt than a GPU, at least for things like image processing.

Re: Dell admits consumers don't care about AI PCs

#155

Post by marcosdumay »

ericmcer wrote: post #125
wavemode wrote: post #44

Earlier quoted context omitted.

> Ideally the alert would only happen if the comment seemed important but it would readily discard short or nonsensical input. That doesn't sound ideal at all. And in fact highlights what's wrong with AI product development nowadays. AI as a tool is wildly popular. Almost everyone in the world uses ChatGPT or knows someone who does. Here's the thing about tools - you use them in a predictable way and they give you a…

But... A lot of stuff you rely on now was probably once distracting and unpredictable. There are a ton of subtle UX behaviors a modern computer is doing that you don't notice, but if they all disappeared and you had to use windows 95 for a week you would miss. That is more what I am advocating for, subtle background UX improvements based on an LLMs ability to interpret a users intent. We had limited abilities to look…

> But... A lot of stuff you rely on now was probably once distracting and unpredictable.

And nobody relied on them when they were distracting and unpredictable. People only rely on them now because they are not.

LLMs won't ever be predictable. They are designed not to be. A predictable AI is something different from a LLM.

Re: Dell admits consumers don't care about AI PCs

#156

Post by zozbot234 »

pseudosavant wrote: post #133

I don't know how many others here have a CoPilot+ PC but the NPU on it is basically useless. There isn't any meaningful feature I get by having that NPU. They are far too limited to ever do any meaningful local LLM inference, image processing or generation. It handles stuff like video chat background blurring, but users' PC's have been doing that for years now without an NPU.

NPUs overall need better support from local AI frameworks. They're not "useless" for what they can do (low-precision bulk compute, which is potentially relevant for many of the newer models) and they could help address thermal limits due to their higher power efficiency compared to CPU/iGPU. but that all requires specialized support that hasn't been coming.

Re: Dell admits consumers don't care about AI PCs

#157

Post by nottorp »

CamperBob2 wrote: post #127
nottorp wrote: post #83

Earlier quoted context omitted.

> I like to know it inside out. Surrendering all that feels alien to me. It's okay, you'll just forget you were ever able to know your code :)

I've already forgotten most assembly languages I ever used. I look forward to forgetting C++.

Last part is very common, but what's wrong with assembly languages?

But I wasn't talking about forgetting one language or another, i was talking about forgetting to program completely.

Re: Dell admits consumers don't care about AI PCs

#158

Post by delecti »

giancarlostoro wrote: post #124
wolvoleo wrote: post #118

Earlier quoted context omitted.

It sounds like you mean STT not TTS there?

You're right, in my rage I typod, its really frustrating, even friends will text me and their text makes no sense, and 2 minutes later "STUPID VOICE TO TEXT" I have a few friends who drive trucks, so they need to be able to use their voice to communicate.

Better speech transcription is cool, but that feels kinda contrived. Phone calls exist, so do voice messages sent via texting apps, and professional drivers can also just wait a bit to send messages if they really must be text; they're on the job, but if it's really that urgent they can pull over.

Re: Dell admits consumers don't care about AI PCs

#159

Post by zozbot234 »

martinald wrote: post #149
kenjackson wrote: post #140

Earlier quoted context omitted.

I'd love to see a thorough breakdown of what these local NPUs can really do. I've had friends ask me about this (as the resident computer expert) and I really have no idea. Everything I see advertised for (blurring, speech to text, etc...) are all things that I never felt like my non-NPU machine struggled with. Is there a single remotely killer application for local client NPUs?

The problem is essentially memory bandiwdth afiak. Simplifying a lot in my reply, but most NPUs (all?) do not have faster memory bandwidth than the GPU. They were originally designed when ML models were megabytes not gigabytes. They have a small amount of very fast SRAM (4MB I want to say?). LLM models _do not_ fit into 4MB of SRAM :). And LLM inference is heavily memory bandwidth bound (reading input tokens isn't th…

NPUs are still useful for LLM pre-processing and other compute-bound tasks. They will waste memory bandwidth during LLM generation phase (even in the best-case scenario where they aren't physically bottlenecked on bandwidth to begin with, compared to the iGPU) since they generally have to read padded/dequantized data from main memory as they compute directly on that, as opposed to being able to unpack it in local registers like iGPUs can.

> usually on device prompts are very short

Sure, but that might change with better NPU support, making time-to-first-token quicker with larger prompts.

Re: Dell admits consumers don't care about AI PCs

#160

Post by Someone »

kenjackson wrote: post #140
pseudosavant wrote: post #133

I don't know how many others here have a CoPilot+ PC but the NPU on it is basically useless. There isn't any meaningful feature I get by having that NPU. They are far too limited to ever do any meaningful local LLM inference, image processing or generation. It handles stuff like video chat background blurring, but users' PC's have been doing that for years now without an NPU.

I'd love to see a thorough breakdown of what these local NPUs can really do. I've had friends ask me about this (as the resident computer expert) and I really have no idea. Everything I see advertised for (blurring, speech to text, etc...) are all things that I never felt like my non-NPU machine struggled with. Is there a single remotely killer application for local client NPUs?

> Everything I see advertised for (blurring, speech to text, etc...) are all things that I never felt like my non-NPU machine struggled with.

I don’t know how good these neural engines are, but transistors are dead-cheap nowadays. That makes adding specialized hardware a valuable option, even if it doesn’t speed up things but ‘only’ decreases latency or power usage.

Post reply on HN