AI PCs Aren't Good at AI: The CPU Beats the NPU
1–10 of 303 posts
Re: AI PCs Aren't Good at AI: The CPU Beats the NPU
#2[deleted]
Re: AI PCs Aren't Good at AI: The CPU Beats the NPU
#3[deleted]
Re: AI PCs Aren't Good at AI: The CPU Beats the NPU
#4The write up on the GitHub repo is much more informative than the blog.
When running int8 matmul using onnx performance is ~0.6TF.
Re: AI PCs Aren't Good at AI: The CPU Beats the NPU
#5In general MAC unit utilization tends to be low for transformers, but 1.3% seems pretty bad. I wonder if they fucked up the memory interface for the NPU. All the MACs in the world are useless if you cannot feed them.
Re: AI PCs Aren't Good at AI: The CPU Beats the NPU
#6I laughed when I saw that the Qualcomm “AI PC” is described as this in the ComfyUI docs:
"Avoid", "Nothing works", "Worthless for any AI use"
Re: AI PCs Aren't Good at AI: The CPU Beats the NPU
#7I laughed when I saw that the Qualcomm “AI PC” is described as this in the ComfyUI docs: "Avoid", "Nothing works", "Worthless for any AI use"
[deleted]
Re: AI PCs Aren't Good at AI: The CPU Beats the NPU
#8These NPUs are tying up a substantial amount of silicon area so it would be a real shame if they end up not being used for much. I can't find a die analysis of the Snapdragon X which isolates the NPU specifically but AMDs equivalent with the same ~50 TOPS performance target can be seen here, and takes up about as much area as three high performance CPU cores:
https://www.techpowerup.com/325035/amd-strix-point-silicon-p...
Re: AI PCs Aren't Good at AI: The CPU Beats the NPU
#9> the 45 trillion operations per second that’s listed in the specs
Such a spec should be ideally be accompanied by code demonstrating or approximating the claimed performance. I can't imagine a sports car advertising a 0-100km/h spec of 2.0 seconds where a user is unable to get below 5 seconds.
Re: AI PCs Aren't Good at AI: The CPU Beats the NPU
#10In general MAC unit utilization tends to be low for transformers, but 1.3% seems pretty bad. I wonder if they fucked up the memory interface for the NPU. All the MACs in the world are useless if you cannot feed them.
I recall looking over the Ryzen AI architecture and the NPU is just plugged into PCIe and thus gets completely crap memory bandwidth. I would expect it might be similar here.