Apple introduces M4 chip • Hacker News Board

Re: Apple introduces M4 chip

#801

Post by pompino » Wed, May 08, 2024, 3:21 AM UTC

andsoitis wrote: post #539

rsp1984 wrote: post #457

Together with next-generation ML accelerators in the CPU, the high-performance GPU, and higher-bandwidth unified memory, the Neural Engine makes M4 an outrageously powerful chip for AI. In case it is not abundantly clear by now: Apple's AI strategy is to put inference (and longer term even learning) on edge devices. This is completely coherent with their privacy-first strategy (which would be at odds with sending dat…

> In case it is not abundantly clear by now: Apple's AI strategy is to put inference (and longer term even learning) on edge devices. This is completely coherent with their privacy-first strategy (which would be at odds with sending data up to the cloud for processing). Their primary business goal is to sell hardware. Yes, they’ve diversified into services and being a shopping mall for all, but it is about selling lu…

>The promise of privacy is one way in which they position themselves, but I would not bet the bank on that being true forever.

They failed with their ad-business so this is a nice pivot. I'll take it, I'm not usually a cheerleader for Apple, but I'll support anyone who can erode Google's surveillance dominance.

Re: Apple introduces M4 chip

#802

Post by mycall » Wed, May 08, 2024, 3:25 AM UTC

lll-o-lll wrote: post #790

deergomoo wrote: post #678

Earlier quoted context omitted.

I've been saying this for years, I would love to get a desktop Mac and use an iPad for the occasional bit of portable development I do away from a desk, like when I want to noodle on an idea in front of the TV. I'm very happy with my MacBook, but I don't like that the mega expensive machine I want to keep for 5+ years needs to be tied to a limited-life lithium battery that's costly and labour intensive to replace, ju…

Is there no RDP equivalent for mac? Just RDP into your main workstation?

https://support.apple.com/en-il/guide/mac-help/mh11848/mac

Re: Apple introduces M4 chip

#803

Post by MBCook » Wed, May 08, 2024, 3:32 AM UTC

tweetle_beetle wrote: post #645

rsp1984 wrote: post #457

Together with next-generation ML accelerators in the CPU, the high-performance GPU, and higher-bandwidth unified memory, the Neural Engine makes M4 an outrageously powerful chip for AI. In case it is not abundantly clear by now: Apple's AI strategy is to put inference (and longer term even learning) on edge devices. This is completely coherent with their privacy-first strategy (which would be at odds with sending dat…

I wonder if BYOE (bring your own electricity) also plays a part in their long term vision? Data centres are expensive in terms of hardware, staffing and energy. Externalising this cost to customers saves money, but also helps to paint a green(washing) narrative. It's more meaningful to more people to say they've cut their energy consumption by x than to say they have a better server obselesence strategy, for example.

I’m not sure it’s that (benched pointed out their carbon commitment) as simple logistics.

Apple doesn’t have to build the data centers. Apple doesn’t have to buy the AI capacity themselves (even if from TSMC for Apple designed chips). Apple doesn’t have to have the personnel for the data centers or the air conditioning. They don’t have to pay for all the network bandwidth.

There are benefits to the user to having the AI run on their own devices in terms of privacy and latency as mentioned by the GP.

But there are also benefits to Apple simply because it means it’s no longer their resources being used up above and beyond electricity.

I keep reading about companies having trouble getting GPUs from the cloud providers and that some crypto networks have pivoted to selling GPU access for AI work as crypto profits fall.

Apple doesn’t have to deal with any of that. They have underused silicon sitting out there ready to light up to make their customers happy (and perhaps interested in buying a faster device).

Re: Apple introduces M4 chip

#804

Post by MBCook » Wed, May 08, 2024, 3:34 AM UTC

avianlyric wrote: post #548

joelthelion wrote: post #461

Earlier quoted context omitted.

>n case it is not abundantly clear by now: Apple's AI strategy is to put inference (and longer term even learning) I'm curious: is anyone seriously using apple hardware to train Ai models at the moment? Obviously not the big players, but I imagine it might be a viable option for Ai engineers in smaller, less ambitious companies.

Apple are. Their “Personal Voice” feature fine tunes a voice model on device using recordings of your own voice. An older example is the “Hey Siri” model, which is fine tuned to your specific voice. But with regards to on device training, I don’t think anyone is seriously looking at training a model from scratch on device, that doesn’t make much sense. But taking models and fine tuning them to specific users makes a…

They already do some “simple” training on device. The example I can think of is photo recognition in the photo library. It likely builds on something else but being able to identify which phase is your grandma versus your neighbor is not done in Apple‘s cloud. It’s done when your devices are idle and plugged into power.

A few years ago it wasn’t shared between devices so each device had to do it themselves. I don’t know if it’s shared at this point.

I agree you’re not going to be training an LLM or anything. But smaller tasks limited and scope may prove a good fit.

Re: Apple introduces M4 chip

#805

Post by tyrd12 » Wed, May 08, 2024, 3:34 AM UTC

praseodym wrote: post #10

"With these improvements to the CPU and GPU, M4 maintains Apple silicon’s industry-leading performance per watt. M4 can deliver the same performance as M2 using just half the power. And compared with the latest PC chip in a thin and light laptop, M4 can deliver the same performance using just a fourth of the power." That's an incredible improvement in just a few years. I wonder how much of that is Apple engineering a…

Considering the cost difference would that still make M4 better. Whatever the savings in power are offset by the price?

Re: Apple introduces M4 chip

#806

Post by MBCook » Wed, May 08, 2024, 3:47 AM UTC

deanishe wrote: post #521

joelthelion wrote: post #461

Earlier quoted context omitted.

>n case it is not abundantly clear by now: Apple's AI strategy is to put inference (and longer term even learning) I'm curious: is anyone seriously using apple hardware to train Ai models at the moment? Obviously not the big players, but I imagine it might be a viable option for Ai engineers in smaller, less ambitious companies.

Isn't Apple hardware too expensive to make that worthwhile?

You want to buy a bunch of new equipment to do training? Yeah Mac’s aren’t going to make sense.

You want your developers to be able to do training locally and they already use Macs? Maybe an upgrade would make business sense. Even if you have beefy servers or the cloud for large jobs.

Re: Apple introduces M4 chip

#807

Post by MBCook » Wed, May 08, 2024, 3:54 AM UTC

aiauthoritydev wrote: post #627

rsp1984 wrote: post #457

Together with next-generation ML accelerators in the CPU, the high-performance GPU, and higher-bandwidth unified memory, the Neural Engine makes M4 an outrageously powerful chip for AI. In case it is not abundantly clear by now: Apple's AI strategy is to put inference (and longer term even learning) on edge devices. This is completely coherent with their privacy-first strategy (which would be at odds with sending dat…

I think these days everyone links their products with AI. Today even BP CEO linked his business with AI. Edge inference and cloud inference are not mutually exclusive choices. Any serious provider will provide both and the improvement in quality of services come from you giving more of your data to the service provider. Most people are totally fine with that and that will not change any time sooner. Privacy paranoia…

I agree. Apple has been on this path for a while, the first processor with a Neural Engine was the A11 in 2017 or so. The path didn’t appear to change at all.

The big differences today that stood out to me were adopting AI as a term (they used machine learning before) and repeating the term AI everywhere they could shove it in since that’s obviously what the street wants to hear.

That’s all that was different. And I’m not surprised they emphasized it given all the weird “Apple is behind on AI“ articles that have been going around.

Re: Apple introduces M4 chip

#808

Post by MBCook » Wed, May 08, 2024, 3:58 AM UTC

aiauthoritydev wrote: post #623

rsp1984 wrote: post #457

Together with next-generation ML accelerators in the CPU, the high-performance GPU, and higher-bandwidth unified memory, the Neural Engine makes M4 an outrageously powerful chip for AI. In case it is not abundantly clear by now: Apple's AI strategy is to put inference (and longer term even learning) on edge devices. This is completely coherent with their privacy-first strategy (which would be at odds with sending dat…

Edge inference and cloud inference are not mutually exclusive and chances are any serious player would be dipping their toes in both.

Right. The difference is that Apple has a ton of edge capacity, they’ve been building it for a long time.

Google and Samsung have been building it too, at different speeds.

Intel and AMD seem further behind (at the moment) unless the user has a strong GPU, which is especially uncommon on the most popular kind of computer: laptops.

And if you’re not one of those four companies… you probably don’t have much capable consumer edge hardware.

Re: Apple introduces M4 chip

#809

Post by the_king » Wed, May 08, 2024, 4:00 AM UTC

rsp1984 wrote: post #457

Together with next-generation ML accelerators in the CPU, the high-performance GPU, and higher-bandwidth unified memory, the Neural Engine makes M4 an outrageously powerful chip for AI. In case it is not abundantly clear by now: Apple's AI strategy is to put inference (and longer term even learning) on edge devices. This is completely coherent with their privacy-first strategy (which would be at odds with sending dat…

I'm all for running as much on the edge as possible, but we're not even close to being able to do real-time inference on Frontier models on Macs or iPads, and that's just for vanilla LLM chatbots. Low-precision Llama 3-8b is awesome, but it isn't a Claude 3 replacer, totally drains my battery, and is slow (M1 Max).

Multimodal agent setups are going to be data center/home-lab only for at least the next five years.

Apple isn't about to put 80GB on VRAM in an iPad for about 15 reasons.

Re: Apple introduces M4 chip

#810

Post by SkyMarshal » Wed, May 08, 2024, 4:00 AM UTC

> M4 has Apple’s fastest Neural Engine ever, capable of up to 38 trillion operations per second, which is faster than the neural processing unit of any AI PC today.

I didn't even realize there is other PC-level hardware with AI-specific compute. What's the AMD and Intel equivalent of Neural Engine? (not that it matters since it seems the GPU where most of the AI workload is handled anyway)