Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AI PCs Aren't Good at AI: The CPU Beats the NPU

github.com

Post reply on HN
Print view View original

61–70 of 303 posts

Re: AI PCs Aren't Good at AI: The CPU Beats the NPU

#61

Post by Dylan16807 »

moffkalast wrote: post #50
eightysixfour wrote: post #42

Earlier quoted context omitted.

The 7940HS shipped before recall and doesn't support it because it is not performant enough, so, that doesn't make sense. I just gave you a use case, mine in particular uses it for background blur and eye contact filters with the webcam and uses essentially no power to do it. If I do the same filters with nvidia broadcast, the power usage is dramatically higher.

I doubt there's no notable power draw, NPUs in general have always pulled a handful of watts which should at least about match a modern CPU's idle draw. But it does seem odd that your power usage doesn't change at all, it might be always powered on or something. Eye contact filters seem like a horrible thing, autocorrect won't work better than a dictionary with a tiny model and I doubt these things can come even clos…

> I doubt these things can come even close to running whisper for decent voice transcription.

Whisper runs almost realtime on a single core of my very old CPU. I'd be very surprised if it can't fit in an NPU.

Re: AI PCs Aren't Good at AI: The CPU Beats the NPU

#62

Post by nl »

renewiltord wrote: post #51
ezst wrote: post #39

Earlier quoted context omitted.

I can't wait for the LLM fad to be over so we get some sanity (and efficiency) back. I personally have no use for this extra hardware ("GenAI" doesn't help me in any way nor supports any work-related tasks). Worse, most people have no use for that (and recent surveys even show predominant hostility towards AI creep). We shouldn't be paying extra for that, it should be opt-in, and then it would become clear (by lookin…

I was telling someone this and they gave me link to a laptop with higher battery life and better performance than my own, but I kept explaining to them that the feature I cared most about was die size. They couldn't understand it so I just had to leave them alone. Non-technical people don't get it. Die size is what I care about. It's a critical feature and so many mainstream companies are missing out on my money beca…

Is this a parody?

Why would anyone care about die size? And if you do why not get one of the many low power laptops with Atoms etc that do have small die size?

Re: AI PCs Aren't Good at AI: The CPU Beats the NPU

#63

Post by a2l3aQ »

IshKebab wrote: post #53
Kon-Peki wrote: post #36

Earlier quoted context omitted.

Modern chips have to dedicate a certain percentage of the die to dark silicon [1] (or else they melt/throttle to uselessness), and these kinds of components count towards that amount. So the point of these components is to be used, but not to be used too much. Instead of an NPU, they could have used those transistors and die space for any number of things. But they wouldn't have put additional high performance CPU co…

If they aren't being used it would be better to dedicate the space to more SRAM.

The point is parts of the CPU have to be off or throttled down when other components are under load to maintain TDP, adding cache that would almost certainly be being used defeats the point of that.

Re: AI PCs Aren't Good at AI: The CPU Beats the NPU

#64

Post by throwaway314155 »

Janicc wrote: post #58
DrillShopper wrote: post #43

Earlier quoted context omitted.

It supports Microsoft's Recall (now required) spyware

Please remind me again how Recall sends data to Microsoft. I must've missed that part. Or are you against the print screen button too? I heard that takes images too. Very scary.

[dead]

Re: AI PCs Aren't Good at AI: The CPU Beats the NPU

#66

Post by dagaci »

jamesy0ung wrote: post #14

What exactly does Windows do with a NPU? I don't own an 'AI PC' but it seems like the NPUs are slow and can't run much. I know Apple's Neural Engine is used to power Face ID and the facial recognition stuff in Photos, among other things.

Its used for improving video calls, special effects, image editing/ effects and noise cancelling, teams stuff

Re: AI PCs Aren't Good at AI: The CPU Beats the NPU

#67

Post by JohnFen »

jsheard wrote: post #8

These NPUs are tying up a substantial amount of silicon area so it would be a real shame if they end up not being used for much. I can't find a die analysis of the Snapdragon X which isolates the NPU specifically but AMDs equivalent with the same ~50 TOPS performance target can be seen here, and takes up about as much area as three high performance CPU cores: https://www.techpowerup.com/325035/amd-strix-point-silicon…

> These NPUs are tying up a substantial amount of silicon area so it would be a real shame if they end up not being used for much.

This has been my thinking. Today you have to go out of your way to buy a system with an NPU, so I don't have any. But tomorrow, will they just be included by default? That seems like a waste for those of us who aren't going to be running models. I wonder what other uses they could be put to?

Re: AI PCs Aren't Good at AI: The CPU Beats the NPU

#68

Post by throwaway48476 »

nl wrote: post #62
renewiltord wrote: post #51

Earlier quoted context omitted.

I was telling someone this and they gave me link to a laptop with higher battery life and better performance than my own, but I kept explaining to them that the feature I cared most about was die size. They couldn't understand it so I just had to leave them alone. Non-technical people don't get it. Die size is what I care about. It's a critical feature and so many mainstream companies are missing out on my money beca…

Is this a parody? Why would anyone care about die size? And if you do why not get one of the many low power laptops with Atoms etc that do have small die size?

Maybe through a game of telephone they confused die size and node size?

Re: AI PCs Aren't Good at AI: The CPU Beats the NPU

#69

Post by jsheard »

a2l3aQ wrote: post #63
IshKebab wrote: post #53

Earlier quoted context omitted.

If they aren't being used it would be better to dedicate the space to more SRAM.

The point is parts of the CPU have to be off or throttled down when other components are under load to maintain TDP, adding cache that would almost certainly be being used defeats the point of that.

Doesn't SRAM have much lower power density than logic with the same area though? Hence why AMD can get away with physically stacking cache on top of more cache in their X3D parts, without the bottom layer melting.

Re: AI PCs Aren't Good at AI: The CPU Beats the NPU

#70

Post by cmeacham98 »

Janicc wrote: post #58
DrillShopper wrote: post #43

Earlier quoted context omitted.

It supports Microsoft's Recall (now required) spyware

Please remind me again how Recall sends data to Microsoft. I must've missed that part. Or are you against the print screen button too? I heard that takes images too. Very scary.

While calling it spyware like GP is over-exaggeration to a ridiculous level, comparing Recall to Print Screen is also inaccurate:

Print Screen takes images on demand, Recall does so effectively at random. This means Recall could inadvertently screenshot and store information you didn't intend to keep a record of (To give an extreme example: Imagine an abuser uses Recall to discover their spouse browsing online domestic violence resources).

Post reply on HN