Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AI PCs Aren't Good at AI: The CPU Beats the NPU

github.com

Post reply on HN
Print view View original

71–80 of 303 posts

Re: AI PCs Aren't Good at AI: The CPU Beats the NPU

#71

Post by jsheard »

JohnFen wrote: post #67
jsheard wrote: post #8

These NPUs are tying up a substantial amount of silicon area so it would be a real shame if they end up not being used for much. I can't find a die analysis of the Snapdragon X which isolates the NPU specifically but AMDs equivalent with the same ~50 TOPS performance target can be seen here, and takes up about as much area as three high performance CPU cores: https://www.techpowerup.com/325035/amd-strix-point-silicon…

> These NPUs are tying up a substantial amount of silicon area so it would be a real shame if they end up not being used for much. This has been my thinking. Today you have to go out of your way to buy a system with an NPU, so I don't have any. But tomorrow, will they just be included by default? That seems like a waste for those of us who aren't going to be running models. I wonder what other uses they could be put…

> But tomorrow, will they just be included by default?

That's already the way things are going due to Microsoft decreeing that Copilot+ is the future of Windows, so AMD and Intel are both putting NPUs which meet the Copilot+ performance standard into every consumer part they make going forwards to secure OEM sales.

Re: AI PCs Aren't Good at AI: The CPU Beats the NPU

#72

Post by numpad0 »

boomskats wrote: post #30
eightysixfour wrote: post #15

I thought the purpose of these things was not to be fast, but to be able to run small models with very little power usage? I have a newer AMD laptop with an NPU, and my power usage doesn't change using the video effects that supposedly run on it, but goes up when using the nvidia studio effects. It seems like the NPUs are for very optimized models that do small tasks, like eye contact, background blur, autocorrect mo…

That's especially true because yours is a Xilinx FPGA. The one that they just attached to the latest gen mobile ryzens is 5x more capable too. AMD are doing some fantastic work at the moment, they just don't seem to be shouting about it. This one is particularly interesting https://lore.kernel.org/lkml/DM6PR12MB3993D5ECA50B27682AEBE1... edit: not an FPGA. TIL. :'(

Sorry for an OT comment but what is going on with that ascii art!? The content fits within 80 columns just fine[1], is it GPT generated?

1: https://pastebin.com/raw/R9BrqETR

Re: AI PCs Aren't Good at AI: The CPU Beats the NPU

#73

Post by thfuran »

nl wrote: post #62
renewiltord wrote: post #51

Earlier quoted context omitted.

I was telling someone this and they gave me link to a laptop with higher battery life and better performance than my own, but I kept explaining to them that the feature I cared most about was die size. They couldn't understand it so I just had to leave them alone. Non-technical people don't get it. Die size is what I care about. It's a critical feature and so many mainstream companies are missing out on my money beca…

Is this a parody? Why would anyone care about die size? And if you do why not get one of the many low power laptops with Atoms etc that do have small die size?

Yes, they're making fun of the comment they replied to.

Re: AI PCs Aren't Good at AI: The CPU Beats the NPU

#74

Post by wtallis »

itishappy wrote: post #23
AlexandrB wrote: post #16

Earlier quoted context omitted.

> Also, people often mistake the reason for an NPU is "speed". That's not correct. The whole point of the NPU is rather to focus on low power consumption. I have a sneaking suspicion that the real real reason for an NPU is marketing. "Oh look, NVDA is worth $3.3T - let's make sure we stick some AI stuff in our products too."

I assume you're both right. I'm sure NPUs exist to fill a very real niche, but I'm also sure they're being shoehorned in everywhere regardless of product fit because "AI big right now."

Looking at it slightly differently: putting low-power NPUs into laptop and phone SoCs is how to get on the AI bandwagon in a way that NVIDIA cannot easily disrupt. There are plenty of systems where a NVIDIA discrete GPU cannot fit into the budget (of $ or Watts). So even if NPUs are still somewhat of a solution in search of a problem (aka a killer app or two), they're not necessarily a sign that these manufacturers are acting entirely without strategy.

Re: AI PCs Aren't Good at AI: The CPU Beats the NPU

#75

Post by jonas21 »

JohnFen wrote: post #67
jsheard wrote: post #8

These NPUs are tying up a substantial amount of silicon area so it would be a real shame if they end up not being used for much. I can't find a die analysis of the Snapdragon X which isolates the NPU specifically but AMDs equivalent with the same ~50 TOPS performance target can be seen here, and takes up about as much area as three high performance CPU cores: https://www.techpowerup.com/325035/amd-strix-point-silicon…

> These NPUs are tying up a substantial amount of silicon area so it would be a real shame if they end up not being used for much. This has been my thinking. Today you have to go out of your way to buy a system with an NPU, so I don't have any. But tomorrow, will they just be included by default? That seems like a waste for those of us who aren't going to be running models. I wonder what other uses they could be put…

NPUs are already included by default in the Apple ecosystem. Nobody seems to mind.

Re: AI PCs Aren't Good at AI: The CPU Beats the NPU

#76

Post by tedunangst »

nl wrote: post #62
renewiltord wrote: post #51

Earlier quoted context omitted.

I was telling someone this and they gave me link to a laptop with higher battery life and better performance than my own, but I kept explaining to them that the feature I cared most about was die size. They couldn't understand it so I just had to leave them alone. Non-technical people don't get it. Die size is what I care about. It's a critical feature and so many mainstream companies are missing out on my money beca…

Is this a parody? Why would anyone care about die size? And if you do why not get one of the many low power laptops with Atoms etc that do have small die size?

No, no, no, you just don't get it. The only thing Dell will sell me is a laptop 324mm wide, which is totally appalling, but if they offered me a laptop that's 320mm wide, I'd immediately buy it. In my line of work, which is totally serious business, every millimeter counts.

Re: AI PCs Aren't Good at AI: The CPU Beats the NPU

#77

Post by Terr_ »

Janicc wrote: post #58
DrillShopper wrote: post #43

Earlier quoted context omitted.

It supports Microsoft's Recall (now required) spyware

Please remind me again how Recall sends data to Microsoft. I must've missed that part. Or are you against the print screen button too? I heard that takes images too. Very scary.

> Please remind me again how Recall sends data to Microsoft. I must've missed that part.

Sure, just post the source code and I'll point out where it does so, I somehow misplaced my copy. /s

The core problem here is trust, and over the last several years Microsoft has burned a hell of a lot of theirs with power-users of Windows. Even their most strident public promises of Recall being "opt-in" and "on-device only" will--paradoxically--only be kept as long as enough people remain suspicious.

Glance away and MS go back to their old games, pushing a mandatory "security update" which reset or entirely-removes your privacy settings and adding new "telemetry" streams which you cannot inspect.

Re: AI PCs Aren't Good at AI: The CPU Beats the NPU

#78

Post by tedunangst »

tromp wrote: post #9

> the 45 trillion operations per second that’s listed in the specs Such a spec should be ideally be accompanied by code demonstrating or approximating the claimed performance. I can't imagine a sports car advertising a 0-100km/h spec of 2.0 seconds where a user is unable to get below 5 seconds.

I have some bad news for you regarding how car acceleration is measured.

Re: AI PCs Aren't Good at AI: The CPU Beats the NPU

#79

Post by JohnFen »

jonas21 wrote: post #75
JohnFen wrote: post #67

Earlier quoted context omitted.

> These NPUs are tying up a substantial amount of silicon area so it would be a real shame if they end up not being used for much. This has been my thinking. Today you have to go out of your way to buy a system with an NPU, so I don't have any. But tomorrow, will they just be included by default? That seems like a waste for those of us who aren't going to be running models. I wonder what other uses they could be put…

NPUs are already included by default in the Apple ecosystem. Nobody seems to mind.

It's not really a question of minding if it's there, unless its presence increases cost, anyway. It just seems a waste to let it go idle, so my mind wanders to what other use I could put that circuitry to.

Re: AI PCs Aren't Good at AI: The CPU Beats the NPU

#80

Post by bloated5048 »

Janicc wrote: post #58
DrillShopper wrote: post #43

Earlier quoted context omitted.

It supports Microsoft's Recall (now required) spyware

Please remind me again how Recall sends data to Microsoft. I must've missed that part. Or are you against the print screen button too? I heard that takes images too. Very scary.

It's always safe to assume it does if it's closed source. I rather be suspicious of big corporations seeking to profit at every step than naive.

Also, it's security risk which already been exploited. Sure, MS fixed it, but can you be certain it won't be exploited some time in the future again?

Post reply on HN