Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Dell admits consumers don't care about AI PCs

pcgamer.com

Post reply on HN
Print view View original

331–340 of 418 posts

Re: Dell admits consumers don't care about AI PCs

#331

Post by andrekandre »

zdragnar wrote: post #251
FridgeSeal wrote: post #233

Earlier quoted context omitted.

It’s also trivially solvable with idk, a length check, or any number of other things which don’t need to 100b parameters to calculate.

This was a problem at my last job. Boss kept suggesting shoving AI into features, and I kept pointing out we could make the features better with less effort using simple heuristics in a few lines of code, and skip adding AI altogether. So much of it nowadays is like the blockchain craze, trying to use it as a solution for every problem until it sticks.

  > Boss kept suggesting shoving AI into features, and I kept pointing out we could make the features better with less effort using simple heuristics in a few lines of code
depending on what it is, it would probably also cost less money (no paying for token usage), use less electricity and be more reliable (less probabilistic, more deterministic), and easier to maintain (just fix the bug in the code vs prompt/input spelunking) as well.

there are definitely useful applications for end user features, but a lot of this is ordered from on-high top-down and product managers need to appease them...

Re: Dell admits consumers don't care about AI PCs

#332

Post by wtallis »

mrinterweb wrote: post #286
hamdingers wrote: post #261

Earlier quoted context omitted.

Correct me if I'm wrong, but I thought everyone was still doing inference on the GPU for Apple silicon.

The Apple M series is SoC. The CPU, GPU, NPU, RAM are all part of the chip.

The RAM is not part of the SoC. It's a bunch of separate commodity RAM dies packaged alongside the SoC.

Re: Dell admits consumers don't care about AI PCs

#333

Post by adgjlsfhk1 »

adastra22 wrote: post #329
aleph_minus_one wrote: post #319

Earlier quoted context omitted.

> NPUs aren't meant for LLMs. There are a lot more neural net tech out there than LLMs. OK, but where can I find demo applications of these that will blow my mind (and make me want to buy a PC with an NPU)?

Apple demonstrates this far better. I use their Photos app to manage my family pictures. I can search my images by visible text, by facial recognition, or by description (vector search). It automatically composes "memories" which are little thematic video slideshows. The FaceTime camera automatically keeps my head in frame, and does software panning and zooming as necessary. Automatic caption generation. This is norm…

I've yet to see any convincing benchmarks showing that NPUs are more efficient than normal GPUs (that don't ignore the possibility of downclocking the GPU to make it run slower but more efficient)

Re: Dell admits consumers don't care about AI PCs

#334

Post by adastra22 »

adgjlsfhk1 wrote: post #333
adastra22 wrote: post #329

Earlier quoted context omitted.

Apple demonstrates this far better. I use their Photos app to manage my family pictures. I can search my images by visible text, by facial recognition, or by description (vector search). It automatically composes "memories" which are little thematic video slideshows. The FaceTime camera automatically keeps my head in frame, and does software panning and zooming as necessary. Automatic caption generation. This is norm…

I've yet to see any convincing benchmarks showing that NPUs are more efficient than normal GPUs (that don't ignore the possibility of downclocking the GPU to make it run slower but more efficient)

NPUs are more energy efficient. There is no doubt that a systolic array uses less watts per computation than a tensor operation on a GPU, for these kinds of natural fit applications.

Are they more performant? Hell no. But if you're going to do the calculation, and if you don't care about latency or throughput (e.g. batched processing of vector encodings), why not use the NPU?

Especially on mobile/edge consumer devices -- laptops or phones.

Re: Dell admits consumers don't care about AI PCs

#335

Post by zamadatix »

dworks wrote: post #297
pseudosavant wrote: post #133

I don't know how many others here have a CoPilot+ PC but the NPU on it is basically useless. There isn't any meaningful feature I get by having that NPU. They are far too limited to ever do any meaningful local LLM inference, image processing or generation. It handles stuff like video chat background blurring, but users' PC's have been doing that for years now without an NPU.

What we want as developers: To be able to implement functionality that utilizes a model for tasks like OCR, visual input and analysis, search or re-ranking etc, without having to implement an LLM API and pay for it. Instead we'd like to offer the functionality to users, possibly at no cost, and use their edge computing capacity to achieve it, by calling local protocols and models. What we want as users: To have advan…

What seems to be delivered by NPUs at this point: filtering background noise from our microphone and blurring our camera using a watt or two less than before.

Re: Dell admits consumers don't care about AI PCs

#336

Post by generalizations »

aleph_minus_one wrote: post #301
generalizations wrote: post #263

Earlier quoted context omitted.

NPUs were pushed by Microsoft, who saw the writing on the wall: AI like chatgpt will dominate the user's experience, edge computing is a huge advantage in that regard, and Apple's hardware can do it. NPUs are basically Microsoft trying to fudge their way to a llamacpp-on-Apple-Silicon experience. Obviously it failed, but they couldn't not try.

> NPUs were pushed by Microsoft, who saw the writing on the wall: AI like chatgpt will dominate the user's experience, edge computing is a huge advantage in that regard Then where is a demo application from Microsoft of a model that I can run locally where my user experience is so much better (faster?) if my computer has an NPU?

I didn't say they succeeded, I said they had no option but to try.

Re: Dell admits consumers don't care about AI PCs

#337

Post by zamadatix »

cheema33 wrote: post #270

I did use whisper last night to get the captions out of a video file. The standard whisper tool from OpenAI uses CPU. It took more than 20 minutes to fully process a video file that was a little more than an hour long. During that time my 20-Core CPU was pegged at 100% utilization and the fan got very loud. I then downloaded an Intel version that used the NPU. CPUs stayed close to 0% and fans remained quiet. Total ta…

If you mean OpenVINO, it uses CPU+GPU+NPU - not just the NPU. On something like a 265K the NPU would only be providing 13 of the 36 total TOPS. Overall, I wish they would just put a few more general compute units in the GPU and have 30 TOPS or something but more overall performance in general.

Re: Dell admits consumers don't care about AI PCs

#338

Post by conductr »

Maxatar wrote: post #182
martinald wrote: post #142

Earlier quoted context omitted.

Tbh it's been the same in Windows PCs since forever. Like MMX in the Pentium 1 days - was marketed as basically essential for anything "multimedia" but provided somewhat between no and minimal speedup (v little software was compiled for it). It's quite similar with Apple's neural engine, which afiak is used very little for LLMs, even for coreML. I know I don't think I ever saw it being used in asitop. And I'm sure wh…

I have to disagree with you about MMX. It's possible a lot of software didn't target it explicitly but on Windows MMX was very widely used as it was integrated into DirectX, ffmpeg, GDI, the initial MP3 libraries (l3codeca which was used by Winamp and other popular MP3 players) and the popular DIVX video codec.

Similar to AI PC's right now, very few consumers cared in late 90s. Majority weren't power users creating/editing videos/audio/graphics. Majority of consumers were just consuming and they never had a need to seek out MMX for that, their main consumption bottleneck was likely bandwidth. If they used MMX indirectly in Winamp or DirectX, they probably had no clue.

Today, typical consumers aren't even using a ton of AI or enough to even make them think to buy specialized hardware for it. Maybe that changes but it's the current state.

Re: Dell admits consumers don't care about AI PCs

#339

Post by bane »

Something I learned on HN years ago was the principle that often something that is riding to the top of the hyper curve is usually not a good product, but a good feature in another product.

At CES this year, one of the things that was noted was that "AI" was not being pushed so much as the product, but "things with AI" or "things powered by AI".

This change in messaging seems to be aligning with other macro movements around AI in the public zeitgeist (as AI continues to later phases of the hyper curve) that the companies' who've gone all-in on AI are struggling to adapt to.

The end-state is to be seen, but it's clear that the present technology around AI has utility, but doesn't seem to have enough utility to lift off the hype curve on an continuously upward slope.

Dell is figuring this out, Microsoft is seeing it in their own metrics, Apple and AWS has more or less dipped toes in the pool...I'd wager that we'll see some wild things in the next few years as these big bets unravel into more prosaic approaches that are more realistically aligned with the utility AI is actually providing.

Re: Dell admits consumers don't care about AI PCs

#340

Post by blep-arsh »

fragmede wrote: post #229
bitwize wrote: post #179

Earlier quoted context omitted.

"WHAT IS MY PURPOSE?" "You multiply matrices of INT8s." "OH... MY... GOD" NPUs really just accelerate low-precision matmuls. A lot of them are based on systolic arrays, which are like a configurable pipeline through which data is "pumped" rather than a general purpose CPU or GPU with random memory access. So they're a bit like the "synergistic" processors in the Cell, in the respect that they accelerate some operatio…

Do compilers know how to take advantage of that, or do programs need code that specifically takes advantage of that?

There are specialized computation kernels compiled for NPUs. A high-level program (that uses ONNX or CoreML, for example) can decide whether to run the computation using CPU code, a GPU kernel, or an NPU kernel or maybe use multiple devices in parallel for different parts of the task, but the low-level code is compiled separately for each kind of hardware. So it's somewhat abstracted and automated by wrapper libraries but still up to the program ultimately.
Post reply on HN