Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AI PCs Aren't Good at AI: The CPU Beats the NPU

github.com

Post reply on HN
Print view View original

191–200 of 303 posts

Re: AI PCs Aren't Good at AI: The CPU Beats the NPU

#191

Post by defrost »

jjmarr wrote: post #188
hakfoo wrote: post #184

Earlier quoted context omitted.

We're still looking for "that is useful". The stuff they've been trying to sell AI to the public with is increasingly looking as absurd as every 1978 "you'll store your recipes on the home computer" argument. AI text became a Human Centipede story: Start with a coherent 10-word sentence, let AI balloon it into five pages of flowery nonsense, send it to someone else, who has their AI smash it back down to 10 meaningfu…

Cringe emo girls are trendy now because the nostalgia cycle is hitting the early 2000s. Your kid would be impressed if you told them you were a goth gf. It's not hard to imagine the same will happen with primitive AIs in the 40s.

Early 2000's ??

Bela Lugosi Died in 1979, and Peter Murphy was onto his next band by 1984.

By 2000 Goth was fully a distant dot in the rear view mirror for the OG's

    In 2002, Murphy released *Dust* with Turkish-Canadian composer and producer Mercan Dede, which utilizes traditional Turkish instrumentation and songwriting, abandoning Murphy's previous pop and rock incarnations, and juxtaposing elements from progressive rock, trance, classical music, and Middle Eastern music, coupled with Dede's trademark atmospheric electronics.
https://www.youtube.com/watch?v=Yy9h2q_dr9k

https://en.wikipedia.org/wiki/Bauhaus_(band)

Re: AI PCs Aren't Good at AI: The CPU Beats the NPU

#192

Post by 0xDEAFBEAD »

waveBidder wrote: post #103
renewiltord wrote: post #51

Earlier quoted context omitted.

I was telling someone this and they gave me link to a laptop with higher battery life and better performance than my own, but I kept explaining to them that the feature I cared most about was die size. They couldn't understand it so I just had to leave them alone. Non-technical people don't get it. Die size is what I care about. It's a critical feature and so many mainstream companies are missing out on my money beca…

your satire is off base enough that people don't understand it's satire.

Says a lot about HN that so many believed he was genuine.

Re: AI PCs Aren't Good at AI: The CPU Beats the NPU

#193

Post by lynguist »

ddingus wrote: post #181
cj wrote: post #163

Earlier quoted context omitted.

Rightly so. The M processor really did completely eliminate all sense of “lag” for basic computing (web browsing, restarting your computer, etc). Everything happens nearly instantly, even on the first generation M1 processor. The experience of “waiting for something to load” went away. Not to mention these machines easily last 5-10 years.

I have a first gen M1 and it holds up very nicely even today. I/O is crazy fast and high compute loads get done efficiently. One can bury the machine and lose very little basic interactivity. That part users really like. Frankly the only downside of the MacBook Air is the tiny storage. The 8GB RAM is actually enough most of the time. But general system storage with only 1/4 TB is cramped consistently. Been thinking a…

Why did you buy a 256GB device for personal use in the first place? Too good of a deal? Or saving these $400 for upgrades for something else?

Re: AI PCs Aren't Good at AI: The CPU Beats the NPU

#194

Post by alisonatwork »

bugbuddy wrote: post #173
nxobject wrote: post #165

Earlier quoted context omitted.

As a very happy M1 Max user (should've shelled out for 64GB of RAM, though, for local LLMs!), I don't look forward to seeing how the Google Workspace/Notions/etc. of the world somehow reintroduce lag back in.

The problem for Intel and AMD is they are stuck with an OS that ships with a lag-inducing Anti-malware suite. I just did a simple git log and it took 2000% longer than usual because the Antivirus was triggered to scan and run a simulation on each machine instruction and byte of data accessed. The commit log window stayed blank waiting to load long enough for me to complete another tiny project. It always ruin my day.

Pro tip: turn off malware scanning in your git repos[0]. There is also the new Dev Drive feature in Windows 11 that makes it even easier for developers (and IT admins) to set this kind of thing up via policies[1].

In companies where I worked where the IT team rolled out "security" software to the Mac-based developers, their computers were not noticeably faster than Windows PCs at all, especially given the majority of containers are still linux/amd64, reflecting the actual deployment environment. Meanwhile Windows also runs on ARM anyway, so it's not really something useful to generalize about.

[0] https://support.microsoft.com/en-us/topic/how-to-add-a-file-...

[1] https://learn.microsoft.com/en-us/windows/dev-drive/

Re: AI PCs Aren't Good at AI: The CPU Beats the NPU

#195

Post by positr0n »

jcgrillo wrote: post #147
Kon-Peki wrote: post #36

Earlier quoted context omitted.

Modern chips have to dedicate a certain percentage of the die to dark silicon [1] (or else they melt/throttle to uselessness), and these kinds of components count towards that amount. So the point of these components is to be used, but not to be used too much. Instead of an NPU, they could have used those transistors and die space for any number of things. But they wouldn't have put additional high performance CPU co…

Question--what's to be lost by making your features sufficiently not dense to allow them to cool at full tilt?

Good discussion on how at multi GHz clock speeds, the speed of light is actually limiting on some circuit design choices: https://news.ycombinator.com/item?id=12384596

Re: AI PCs Aren't Good at AI: The CPU Beats the NPU

#196

Post by pjmlp »

Spooky23 wrote: post #159
AlexandrB wrote: post #16

Earlier quoted context omitted.

> Also, people often mistake the reason for an NPU is "speed". That's not correct. The whole point of the NPU is rather to focus on low power consumption. I have a sneaking suspicion that the real real reason for an NPU is marketing. "Oh look, NVDA is worth $3.3T - let's make sure we stick some AI stuff in our products too."

Microsoft needs to throw something in the gap to slow down MacBook attrition. The M processors changed the game. My teams support 250k users. I went from 50 MacBooks in 2020 to over 10,000 today. I added zero staff - we manage them like iPhones.

Microsoft has indeed a problem, however only in countries whose people can afford Apple level prices, and not everyone is a G7 citizen.

Re: AI PCs Aren't Good at AI: The CPU Beats the NPU

#197

Post by djur »

nxobject wrote: post #165
cj wrote: post #163

Earlier quoted context omitted.

Rightly so. The M processor really did completely eliminate all sense of “lag” for basic computing (web browsing, restarting your computer, etc). Everything happens nearly instantly, even on the first generation M1 processor. The experience of “waiting for something to load” went away. Not to mention these machines easily last 5-10 years.

As a very happy M1 Max user (should've shelled out for 64GB of RAM, though, for local LLMs!), I don't look forward to seeing how the Google Workspace/Notions/etc. of the world somehow reintroduce lag back in.

Oh, just work for a company that uses Crowdstrike or similar. You'll get back all the lag you want.

Re: AI PCs Aren't Good at AI: The CPU Beats the NPU

#198

Post by Someone »

j16sdiz wrote: post #129
acchow wrote: post #100

Earlier quoted context omitted.

I was curious so looked it up. Apple's first chip with an NPU was the A11 bionic in Sept 2017. Rockchip's was the RK1808 in Sept 2019.

Google TPU was introduced around same time as apple. Basically everybody knew it can be something around that time, just don't know exactly how

https://en.wikipedia.org/wiki/Tensor_Processing_Unit#Product... shows the first one is from 2015 (publicly announced in 2016). It also shows they have a TDP of 75+W.

I can’t find TDP for Apple’s Neural Engine (https://en.wikipedia.org/wiki/Neural_Engine), but the first version shipped in the iPhone 8, which has a 7 Wh battery, so these are targeting different markets.

Re: AI PCs Aren't Good at AI: The CPU Beats the NPU

#199

Post by djur »

defrost wrote: post #191
jjmarr wrote: post #188

Earlier quoted context omitted.

Cringe emo girls are trendy now because the nostalgia cycle is hitting the early 2000s. Your kid would be impressed if you told them you were a goth gf. It's not hard to imagine the same will happen with primitive AIs in the 40s.

Early 2000's ?? Bela Lugosi Died in 1979, and Peter Murphy was onto his next band by 1984. By 2000 Goth was fully a distant dot in the rear view mirror for the OG's In 2002, Murphy released *Dust* with Turkish-Canadian composer and producer Mercan Dede, which utilizes traditional Turkish instrumentation and songwriting, abandoning Murphy's previous pop and rock incarnations, and juxtaposing elements from progressive…

I'm not sure what "gothic music existed in the 1980s" is meant to indicate as a response to "goths existed in the early 2000s as a cultural archetype".

Re: AI PCs Aren't Good at AI: The CPU Beats the NPU

#200

Post by defrost »

djur wrote: post #199
defrost wrote: post #191

Earlier quoted context omitted.

Early 2000's ?? Bela Lugosi Died in 1979, and Peter Murphy was onto his next band by 1984. By 2000 Goth was fully a distant dot in the rear view mirror for the OG's In 2002, Murphy released *Dust* with Turkish-Canadian composer and producer Mercan Dede, which utilizes traditional Turkish instrumentation and songwriting, abandoning Murphy's previous pop and rock incarnations, and juxtaposing elements from progressive…

I'm not sure what "gothic music existed in the 1980s" is meant to indicate as a response to "goths existed in the early 2000s as a cultural archetype".

That Goths in 2000's were at best second wave nostalgia cycle of Goths from the 1980s.

That people recalling Goths in that period should beware of thinking that was a source and not an echo.

In 2006 Noel Fielding's Richmond Felicity Avenal was a basement dwelling leftover from many years past.

Post reply on HN