Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Dell admits consumers don't care about AI PCs

pcgamer.com

Post reply on HN
Print view View original

211–220 of 418 posts

Re: Dell admits consumers don't care about AI PCs

#211

Post by giantrobot »

martinald wrote: post #142
disfictional wrote: post #132

As someone who spent a year writing an SDK specifically for AI PCs, it always felt like a solution in search of a problem. Like watching dancers in bunny suits sell CPUs, if the consumer doesn't know the pain point you're fixing, they won't buy your product.

Tbh it's been the same in Windows PCs since forever. Like MMX in the Pentium 1 days - was marketed as basically essential for anything "multimedia" but provided somewhat between no and minimal speedup (v little software was compiled for it). It's quite similar with Apple's neural engine, which afiak is used very little for LLMs, even for coreML. I know I don't think I ever saw it being used in asitop. And I'm sure wh…

Apple's neural engine is used a lot by the non-LLM ML tasks all over the system like facial recognition in photos and the like. The point of it isn't to be some beefy AI co-processor but to be a low-power accelerator for background ML workloads.

The same workloads could use the GPU but it's more general purpose and thus uses more power for the same task. The same reason macOS uses hardware acceleration for video codecs and even JPEG, the work could be done on the CPU but cost more in terms of power. Using hardware acceleration helps with the 10+ hour lifetime on the battery.

Re: Dell admits consumers don't care about AI PCs

#212

Post by fckgw »

9rx wrote: post #146
wrl wrote: post #141

Earlier quoted context omitted.

> Ideally the alert would only happen if the comment seemed important but it would readily discard short or nonsensical input. That is really difficult to do in traditional software but is something an LLM could do with low effort. I read this post yesterday and this specific example kept coming back to me because something about it just didn't sit right. And I finally figured it out: Glancing at the alert box (or th…

The problem isn't so much the five seconds, it is the muscle memory. You become accustomed to blindly hitting "Yes" every time you've accidentally typed something into the text box, and then that time when you actually put a lot of effort into something... Boom. Its gone. I have been bitten before. Something like the parent described would be a huge improvement. Granted, it seems the even better UX is to save what th…

Which is fine! That's me making the explicit choice that yes, I want to close this box and yes, I want to lose this data. I don't need an AI evaluating how important it thinks I am and second guessing my judgement call.

I tell the computer what to do, not the other way around.

Re: Dell admits consumers don't care about AI PCs

#213

Post by tliltocatl »

eterm wrote: post #204
mjfisher wrote: post #137

Earlier quoted context omitted.

Serious question: what are those things from windows 95/98 I might miss? Rose tinted glasses perhaps, but I remember it as a very straightforward and consistent UI that provided great feedback, was snappy and did everything I needed. Up to and including little hints for power users like underlining shortcut letters for the & key.

The only one I can think of, literally the only one, is grouped icons. And even that's only because browsers ended up in a weird "windows but tabs but actually tabs are windows" state. So yeah, I'd miss the UX of dragging tabs into their own separate windows. But even that is something that still feels janky in most apps ( windows terminal somehow makes this feel bad, even VS code took a long time to make it feel oka…

It's not like grouped icons wasn't technically infeasible on win95. And honestly, whatever they are more useful is quite debatable. And personally, I don't even have a task panel anymore.

The real stuff not on Win95 that everyone would miss is scalable interfaces/high DPI (not necessary as in HiDPI, just above 640x480). And this one does require A LOT of resources and is still wobbly.

Re: Dell admits consumers don't care about AI PCs

#214

Post by rcxdude »

kenjackson wrote: post #140
pseudosavant wrote: post #133

I don't know how many others here have a CoPilot+ PC but the NPU on it is basically useless. There isn't any meaningful feature I get by having that NPU. They are far too limited to ever do any meaningful local LLM inference, image processing or generation. It handles stuff like video chat background blurring, but users' PC's have been doing that for years now without an NPU.

I'd love to see a thorough breakdown of what these local NPUs can really do. I've had friends ask me about this (as the resident computer expert) and I really have no idea. Everything I see advertised for (blurring, speech to text, etc...) are all things that I never felt like my non-NPU machine struggled with. Is there a single remotely killer application for local client NPUs?

I think a lot of it is just power savings on those features, since the dedicated silicon can be a lot more energy efficient even if it's not much more powerful.

Re: Dell admits consumers don't care about AI PCs

#215

Post by buildbot »

martinald wrote: post #142
disfictional wrote: post #132

As someone who spent a year writing an SDK specifically for AI PCs, it always felt like a solution in search of a problem. Like watching dancers in bunny suits sell CPUs, if the consumer doesn't know the pain point you're fixing, they won't buy your product.

Tbh it's been the same in Windows PCs since forever. Like MMX in the Pentium 1 days - was marketed as basically essential for anything "multimedia" but provided somewhat between no and minimal speedup (v little software was compiled for it). It's quite similar with Apple's neural engine, which afiak is used very little for LLMs, even for coreML. I know I don't think I ever saw it being used in asitop. And I'm sure wh…

Using VisionOCR stuff on MacOS spins my M4 ANE up from 0 to 1W according to poweranalyzer

Re: Dell admits consumers don't care about AI PCs

#216

Post by mrinterweb »

hacker_homie wrote: post #209
pseudosavant wrote: post #133

I don't know how many others here have a CoPilot+ PC but the NPU on it is basically useless. There isn't any meaningful feature I get by having that NPU. They are far too limited to ever do any meaningful local LLM inference, image processing or generation. It handles stuff like video chat background blurring, but users' PC's have been doing that for years now without an NPU.

Yeah, that's because the original npus were a rush job, the amd AI Max is the only one that's worth anything in my opinion.

The Apple M series chips are solid for inference.

Re: Dell admits consumers don't care about AI PCs

#217

Post by convivialdingo »

disfictional wrote: post #132

As someone who spent a year writing an SDK specifically for AI PCs, it always felt like a solution in search of a problem. Like watching dancers in bunny suits sell CPUs, if the consumer doesn't know the pain point you're fixing, they won't buy your product.

I spent a few months working on different edge compute NPUs (ARM mostly) with CNN models and it was really painful. A lot of impressive hardware, but I was always running into software fallbacks for models, custom half-baked NN formats, random caveats, and bad quantization.

In the end it was faster, cheaper, and more reliable to buy a fat server running our models and pay the bandwidth tax.

Re: Dell admits consumers don't care about AI PCs

#218

Post by observationist »

zozbot234 wrote: post #159
martinald wrote: post #149

Earlier quoted context omitted.

The problem is essentially memory bandiwdth afiak. Simplifying a lot in my reply, but most NPUs (all?) do not have faster memory bandwidth than the GPU. They were originally designed when ML models were megabytes not gigabytes. They have a small amount of very fast SRAM (4MB I want to say?). LLM models _do not_ fit into 4MB of SRAM :). And LLM inference is heavily memory bandwidth bound (reading input tokens isn't th…

NPUs are still useful for LLM pre-processing and other compute-bound tasks. They will waste memory bandwidth during LLM generation phase (even in the best-case scenario where they aren't physically bottlenecked on bandwidth to begin with, compared to the iGPU) since they generally have to read padded/dequantized data from main memory as they compute directly on that, as opposed to being able to unpack it in local reg…

The idea that tokenization is what they're for is absurd - you're talking a tenth of a thousandth of a millionth of a percent of efficiency gain in real world usage, if that, and only if someone bothers to implement it in software that actually gets used.

NPUs are racing stripes, nothing more. No killer features or utility, they probably just had stock and a good deal they could market and tap into the AI wave with.

Re: Dell admits consumers don't care about AI PCs

#219

Post by ori_b »

fragmede wrote: post #92
slg wrote: post #42

Earlier quoted context omitted.

>For example, if you close a youtube browser tab with a comment half written it will pop up an `alert("You will lose your comment if you close this window")`. It does this if the comment is a 2 page essay or "asdfasdf". Ideally the alert would only happen if the comment seemed important but it would readily discard short or nonsensical input. That is really difficult to do in traditional software but is something an…

I've been totally AI-pilled because I don't see why that's of questionable utility. How is a regexp going to tell the difference between "asdffghjjk" and "So, she cheated on me". A mere byte count isn't going to do it either. If the computer can tell the difference and be less annoying, it seems useful to me?

Because the computer behaving differently in different circumstances is annoying, especially when there's no clear cue to the user what the hidden knobs that control the circumstances are.
Post reply on HN