Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AI PCs Aren't Good at AI: The CPU Beats the NPU

github.com

Post reply on HN
Print view View original

41–50 of 303 posts

Re: AI PCs Aren't Good at AI: The CPU Beats the NPU

#41

Post by DrillShopper »

ezst wrote: post #39
jsheard wrote: post #8

These NPUs are tying up a substantial amount of silicon area so it would be a real shame if they end up not being used for much. I can't find a die analysis of the Snapdragon X which isolates the NPU specifically but AMDs equivalent with the same ~50 TOPS performance target can be seen here, and takes up about as much area as three high performance CPU cores: https://www.techpowerup.com/325035/amd-strix-point-silicon…

I can't wait for the LLM fad to be over so we get some sanity (and efficiency) back. I personally have no use for this extra hardware ("GenAI" doesn't help me in any way nor supports any work-related tasks). Worse, most people have no use for that (and recent surveys even show predominant hostility towards AI creep). We shouldn't be paying extra for that, it should be opt-in, and then it would become clear (by lookin…

Corporatized gains in the market from hype Socialized losses in increased carbon emissions, upheaval from job loss, and higher prices on hardware.

The more they say the future will be better the more that it looks like the status quo.

Re: AI PCs Aren't Good at AI: The CPU Beats the NPU

#42

Post by eightysixfour »

moffkalast wrote: post #38
eightysixfour wrote: post #15

I thought the purpose of these things was not to be fast, but to be able to run small models with very little power usage? I have a newer AMD laptop with an NPU, and my power usage doesn't change using the video effects that supposedly run on it, but goes up when using the nvidia studio effects. It seems like the NPUs are for very optimized models that do small tasks, like eye contact, background blur, autocorrect mo…

[flagged]

The 7940HS shipped before recall and doesn't support it because it is not performant enough, so, that doesn't make sense.

I just gave you a use case, mine in particular uses it for background blur and eye contact filters with the webcam and uses essentially no power to do it. If I do the same filters with nvidia broadcast, the power usage is dramatically higher.

Re: AI PCs Aren't Good at AI: The CPU Beats the NPU

#43

Post by DrillShopper »

jamesy0ung wrote: post #14

What exactly does Windows do with a NPU? I don't own an 'AI PC' but it seems like the NPUs are slow and can't run much. I know Apple's Neural Engine is used to power Face ID and the facial recognition stuff in Photos, among other things.

It supports Microsoft's Recall (now required) spyware

Re: AI PCs Aren't Good at AI: The CPU Beats the NPU

#44

Post by wtallis »

eightysixfour wrote: post #40
errantspark wrote: post #34

Earlier quoted context omitted.

Wait sorry back up a bit here. I can buy a laptop that has a daughter FPGA in it? Does it have GPIO??? Are we seriously building hardware worth buying again in 2024? Do you have a link?

It isn't as fun as you think - they are setup for specific use cases and quite small. Here's a link to the software page: https://ryzenai.docs.amd.com/en/latest/index.html The teeny-tiny "NPU," which is actually an FPGA, is 10 TOPS. Edit: I've been corrected, not an FPGA, just an IP block from Xilinx.

It's not a FPGA. It's an NPU IP block from the Xilinx side of the company. It was presumably originally developed to be run on a Xilinx FPGA, but that doesn't mean AMD did the stupid thing and actually fabbed a FPGA fabric instead of properly synthesizing the design for their laptop ASIC. Xilinx involvement does not automatically mean it's an FPGA.

Re: AI PCs Aren't Good at AI: The CPU Beats the NPU

#45

Post by pclmulqdq »

boomskats wrote: post #30
eightysixfour wrote: post #15

I thought the purpose of these things was not to be fast, but to be able to run small models with very little power usage? I have a newer AMD laptop with an NPU, and my power usage doesn't change using the video effects that supposedly run on it, but goes up when using the nvidia studio effects. It seems like the NPUs are for very optimized models that do small tasks, like eye contact, background blur, autocorrect mo…

That's especially true because yours is a Xilinx FPGA. The one that they just attached to the latest gen mobile ryzens is 5x more capable too. AMD are doing some fantastic work at the moment, they just don't seem to be shouting about it. This one is particularly interesting https://lore.kernel.org/lkml/DM6PR12MB3993D5ECA50B27682AEBE1... edit: not an FPGA. TIL. :'(

It's not an FPGA. It's a VLIW DSP that Xilinx built to go into an FPGA-SoC to help run ML models.

Re: AI PCs Aren't Good at AI: The CPU Beats the NPU

#46

Post by wtallis »

eightysixfour wrote: post #42
moffkalast wrote: post #38

Earlier quoted context omitted.

[flagged]

The 7940HS shipped before recall and doesn't support it because it is not performant enough, so, that doesn't make sense. I just gave you a use case, mine in particular uses it for background blur and eye contact filters with the webcam and uses essentially no power to do it. If I do the same filters with nvidia broadcast, the power usage is dramatically higher.

Intel is also about to launch their first desktop processors with an NPU which falls far short of Microsoft's performance requirements for a "Copilot+ PC". Should still be plenty for webcam use.

Re: AI PCs Aren't Good at AI: The CPU Beats the NPU

#47

Post by boomskats »

eightysixfour wrote: post #40
errantspark wrote: post #34

Earlier quoted context omitted.

Wait sorry back up a bit here. I can buy a laptop that has a daughter FPGA in it? Does it have GPIO??? Are we seriously building hardware worth buying again in 2024? Do you have a link?

It isn't as fun as you think - they are setup for specific use cases and quite small. Here's a link to the software page: https://ryzenai.docs.amd.com/en/latest/index.html The teeny-tiny "NPU," which is actually an FPGA, is 10 TOPS. Edit: I've been corrected, not an FPGA, just an IP block from Xilinx.

Yes, the one on the ryzen 7000 chips like the 7840u isn't massive, but that's the last gen model. The one they've just released with the HX370 chip is estimated at 50 TOPS, which is better than Qualcomm's ARM flagship that this post is about. It's a fivefold improvement in a single generation, it's pretty exciting.

A̵n̵d̵ ̵i̵t̵'̵s̵ ̵a̵n̵ ̵F̵P̵G̵A̵ It's not an FPGA

Re: AI PCs Aren't Good at AI: The CPU Beats the NPU

#48

Post by eightysixfour »

wtallis wrote: post #44
eightysixfour wrote: post #40

Earlier quoted context omitted.

It isn't as fun as you think - they are setup for specific use cases and quite small. Here's a link to the software page: https://ryzenai.docs.amd.com/en/latest/index.html The teeny-tiny "NPU," which is actually an FPGA, is 10 TOPS. Edit: I've been corrected, not an FPGA, just an IP block from Xilinx.

It's not a FPGA. It's an NPU IP block from the Xilinx side of the company. It was presumably originally developed to be run on a Xilinx FPGA, but that doesn't mean AMD did the stupid thing and actually fabbed a FPGA fabric instead of properly synthesizing the design for their laptop ASIC. Xilinx involvement does not automatically mean it's an FPGA.

Thanks for the correction, edited.

Re: AI PCs Aren't Good at AI: The CPU Beats the NPU

#49

Post by boomskats »

wtallis wrote: post #44
eightysixfour wrote: post #40

Earlier quoted context omitted.

It isn't as fun as you think - they are setup for specific use cases and quite small. Here's a link to the software page: https://ryzenai.docs.amd.com/en/latest/index.html The teeny-tiny "NPU," which is actually an FPGA, is 10 TOPS. Edit: I've been corrected, not an FPGA, just an IP block from Xilinx.

It's not a FPGA. It's an NPU IP block from the Xilinx side of the company. It was presumably originally developed to be run on a Xilinx FPGA, but that doesn't mean AMD did the stupid thing and actually fabbed a FPGA fabric instead of properly synthesizing the design for their laptop ASIC. Xilinx involvement does not automatically mean it's an FPGA.

Do you have any more reading on this? How come the XDNA drivers depend on Xilinx' XRT runtime?

Re: AI PCs Aren't Good at AI: The CPU Beats the NPU

#50

Post by moffkalast »

eightysixfour wrote: post #42
moffkalast wrote: post #38

Earlier quoted context omitted.

[flagged]

The 7940HS shipped before recall and doesn't support it because it is not performant enough, so, that doesn't make sense. I just gave you a use case, mine in particular uses it for background blur and eye contact filters with the webcam and uses essentially no power to do it. If I do the same filters with nvidia broadcast, the power usage is dramatically higher.

I doubt there's no notable power draw, NPUs in general have always pulled a handful of watts which should at least about match a modern CPU's idle draw. But it does seem odd that your power usage doesn't change at all, it might be always powered on or something.

Eye contact filters seem like a horrible thing, autocorrect won't work better than a dictionary with a tiny model and I doubt these things can come even close to running whisper for decent voice transcription. Background blur alright, but that's kind of stretching it. I always figured Zoom/Teams do these things serverside anyway.

And alright, if it's not MS making them do it, then they're just chasing the fad themselves while also shipping subpar hardware. Not sure if that makes it better.

Post reply on HN