Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Apple introduces M4 chip

apple.com

Post reply on HN
Print view View original

721–730 of 1001 posts

Re: Apple introduces M4 chip

#721

Post by kllrnohj »

Aurornis wrote: post #371
NorwegianDude wrote: post #264

> M4 has Apple’s fastest Neural Engine ever, capable of up to 38 trillion operations per second, which is faster than the neural processing unit of any AI PC today. I always wonder what crazy meds Apple employees are on. Two RTX 4090s is quite common for hobbyist use, and that is 1321 TOPS each, making two over 69 times more than what Apple claims to be the fastest in the world. That performance is literally less tha…

It's a marketing trick. They're talking about NPU s specifically, which haven't really been rolled out on the PC side. So while they're significantly slower than even casual gaming GPUs, they're technically the fastest NPUs on the market. It's marketing speak.

It's still lower than Qualcomm's Snapdragon Elite X (45 TOPS)

Re: Apple introduces M4 chip

#722

Post by zachbee »

rsp1984 wrote: post #457

Together with next-generation ML accelerators in the CPU, the high-performance GPU, and higher-bandwidth unified memory, the Neural Engine makes M4 an outrageously powerful chip for AI. In case it is not abundantly clear by now: Apple's AI strategy is to put inference (and longer term even learning) on edge devices. This is completely coherent with their privacy-first strategy (which would be at odds with sending dat…

If they're doing inference on edge devices, one challenge I see is protecting model weights. If you want to deploy a proprietary model on an edge AI chip, the weights can get stolen via side-channel attacks [1]. Obviously this isn't a concern for open models, but I doubt Apple would go the open models route.

[1] https://spectrum.ieee.org/how-prevent-ai-power-usage-secrets

Re: Apple introduces M4 chip

#723

Post by 1024core »

> capable of up to 38 trillion operations per second

Assuming these are BF16 ops, by comparison, an H100 from NVIDIA will do 1979 teraFLOPS BF16.

So this "Neural Engine" from Apple is 50x slower than an H100 PCIe version.

Re: Apple introduces M4 chip

#724

Post by _akhe »

JodieBenitez wrote: post #255

And here I am with my MB Air M1 with no plan to upgrade whatsoever because I don't need to... (yes, I understand this is about iPad, but I guess we'll see these M4 on the MB Air as well ?)

You'll want to upgrade to produce generative content - you just don't know it yet.

Re: Apple introduces M4 chip

#725

Post by VelesDude »

andsoitis wrote: post #539
rsp1984 wrote: post #457

Together with next-generation ML accelerators in the CPU, the high-performance GPU, and higher-bandwidth unified memory, the Neural Engine makes M4 an outrageously powerful chip for AI. In case it is not abundantly clear by now: Apple's AI strategy is to put inference (and longer term even learning) on edge devices. This is completely coherent with their privacy-first strategy (which would be at odds with sending dat…

> In case it is not abundantly clear by now: Apple's AI strategy is to put inference (and longer term even learning) on edge devices. This is completely coherent with their privacy-first strategy (which would be at odds with sending data up to the cloud for processing). Their primary business goal is to sell hardware. Yes, they’ve diversified into services and being a shopping mall for all, but it is about selling lu…

I think it was Paul Thurrott on Windows Weekly podcast who said that all these companies don't really care about privacy. Apple takes billions of dollar a year to direct data towards Google via the search defaults. Clearly privacy has a price. And I suspect it will only get worse with time as they keep chasing the next quarter.

Tim Cook unfortunately is so captured in that quarterly mindset of 'please the share holders' that it is only a matter of time.

Re: Apple introduces M4 chip

#727

Post by whimsicalism »

jonplackett wrote: post #699
rsp1984 wrote: post #457

Together with next-generation ML accelerators in the CPU, the high-performance GPU, and higher-bandwidth unified memory, the Neural Engine makes M4 an outrageously powerful chip for AI. In case it is not abundantly clear by now: Apple's AI strategy is to put inference (and longer term even learning) on edge devices. This is completely coherent with their privacy-first strategy (which would be at odds with sending dat…

I think it will be a winning strategy. Lag is a real killer for LLMs. I think they'll have another LLM on a server (maybe a deal for openai/gemini) that the one on the device can use like ChatGPT uses plugins. But on device Apple have a gigantic advantage. Rabbit and Humane are good ideas humbled by shitty hardware that runs out of battery, gets too hot, has to connect to the internet to do literally anything. Apple…

Having used groq and other fast LLM services a fair bit, lag seems negligible. You're literally just passing text at close to the speed of light.

Re: Apple introduces M4 chip

#728

Post by whimsicalism »

threeseed wrote: post #700
jonplackett wrote: post #699

Earlier quoted context omitted.

I think it will be a winning strategy. Lag is a real killer for LLMs. I think they'll have another LLM on a server (maybe a deal for openai/gemini) that the one on the device can use like ChatGPT uses plugins. But on device Apple have a gigantic advantage. Rabbit and Humane are good ideas humbled by shitty hardware that runs out of battery, gets too hot, has to connect to the internet to do literally anything. Apple…

There really isn't enough emphasis on the downsides of server side platforms. So many of these are only deployed in US and so if you're say in country Australia not only do you have all your traffic going to the US but it will be via slow and intermittent cellular connections. It makes using services like LLMs unusably slow. I miss the 90s and having applications and data reside locally.

Unusably slow? It's like 0.3 seconds to first token and then pretty much all of the tokens can follow within a second.

I find it hard to understand the edge usecase for text-based models.

Re: Apple introduces M4 chip

#729

Post by _akhe »

GalaxyNova wrote: post #604

I hate how apple tends to make statements about their products without clear benchmarks.

It's a 10-core CPU + 10-core GPU + 16-core "NPU" (neural processing unit) for AI all on a consumer handheld. It's like a Ferrari engine in a Honda Civic - all we know is it's going to be fast and hopefully it doesn't catch on fire.
Post reply on HN