Apple M3 Ultra • Hacker News Board

Re: Apple M3 Ultra

#51

Post by pier25 » Wed, Mar 05, 2025, 2:34 PM UTC

dwighttk wrote: post #32

ksec wrote: post #22

Previous model of M2 Ultra had max memory of 192GB. Or 128GB for Pro and some other M3 model, which I think is plenty for even 99.9% of professional task. They now bump it to 512GB . Along with insane price tag of $9499 for 512GB Mac Studio. I am pretty sure this is some AI Gold rush.

Maybe .1% of tasks need this RAM, why are they charging so much?

Because the minority that needs that much RAM can't work without it.

In the media composing world they use huge orchestral templates with hundreds and hundreds of tracks with millions of samples loaded into memory.

Re: Apple M3 Ultra

#52

Post by qwertox » Wed, Mar 05, 2025, 2:34 PM UTC

iambateman wrote: post #14

People who know more than me: they’re talking a lot about RAM and not much about GPU. Do you expect this will be able to handle AI workloads well? All I’ve heard for the past two years is how important a beefy GPU is. Curious if that holds true here too.

A beefy GPU which can't hold models in VRAM is of very limited use. You'll see 16 GB of VRAM on gamer Nvidia cards, the RTX 5090 being an exception with 32 GB VRAM. The professional cards have around 96 GB of VRAM.

The thing with these Apple chips is that they have unified memory, where CPU and GPU use the same memory chips, which means that you can load huge models into RAM (no longer VRAM, because that doesn't exist on those devices). And while Apple's integrated GPU isn't as powerful as an Nvidia GPU, it is powerful enough for non-professional workloads and has the huge benefit of access to lots of memory.

Re: Apple M3 Ultra

#53

Post by A4ET8a8uTh0_v2 » Wed, Mar 05, 2025, 2:34 PM UTC

dwighttk wrote: post #32

ksec wrote: post #22

Previous model of M2 Ultra had max memory of 192GB. Or 128GB for Pro and some other M3 model, which I think is plenty for even 99.9% of professional task. They now bump it to 512GB . Along with insane price tag of $9499 for 512GB Mac Studio. I am pretty sure this is some AI Gold rush.

Maybe .1% of tasks need this RAM, why are they charging so much?

I think the answer is because they can ( there is a market for it ). The benefit to a crazy person like me that with this addition, I might be able to grab 128gb version at a lower price.

Re: Apple M3 Ultra

#54

Post by znpy » Wed, Mar 05, 2025, 2:34 PM UTC

dwighttk wrote: post #32

ksec wrote: post #22

Previous model of M2 Ultra had max memory of 192GB. Or 128GB for Pro and some other M3 model, which I think is plenty for even 99.9% of professional task. They now bump it to 512GB . Along with insane price tag of $9499 for 512GB Mac Studio. I am pretty sure this is some AI Gold rush.

Maybe .1% of tasks need this RAM, why are they charging so much?

because they know there will be a large amount of people that don't know this much ram but they'll buy it anyway.

Re: Apple M3 Ultra

#55

Post by InTheArena » Wed, Mar 05, 2025, 2:35 PM UTC

ksec wrote: post #22

Previous model of M2 Ultra had max memory of 192GB. Or 128GB for Pro and some other M3 model, which I think is plenty for even 99.9% of professional task. They now bump it to 512GB . Along with insane price tag of $9499 for 512GB Mac Studio. I am pretty sure this is some AI Gold rush.

Every single AI shop on the planet is trying to figure out if there is enough compute or not to make this a reasonable AI path. If the answer is yes, that 10k is a absolute bargain.

Re: Apple M3 Ultra

#56

Post by nordsieck » Wed, Mar 05, 2025, 2:35 PM UTC

reaperducer wrote: post #19

datadrivenangel wrote: post #5

Unclear what devices this will be in outside of the mac studio. Also most of the comparisons were with M1 and M2 chips, not M4.

most of the comparisons were with M1 and M2 chips, not M4. Is anyone other than a vanishingly small number of hard core hobbiests going to upgrade from an M4 to an M4 Ultra?

> Is anyone other than a vanishingly small number of hard core hobbiests going to upgrade from an M4 to an M4 Ultra?

I expect that the 2 biggest buyers of M4 Ultra will be people who want to run LLMs locally, and people who want the highest performance machine they can get (professionals), but are wedded to mac-only software.

Re: Apple M3 Ultra

#57

Post by klausa » Wed, Mar 05, 2025, 2:35 PM UTC

georgeburdell wrote: post #38

klausa wrote: post #33

Earlier quoted context omitted.

It's not soldered, it's _on the package_ with the SoC.

Probably on package at best

Right, yes, sorry for imprecise language!

Re: Apple M3 Ultra

#58

Post by Numerlor » Wed, Mar 05, 2025, 2:36 PM UTC

InTheArena wrote: post #45

TheTxT wrote: post #8

512GB unified memory is absolutely wild for AI stuff! Compared to how many NVIDIA GPUs you would need, the pricing looks almost reasonable.

A server with 512GB of high-bandwidth GPU addressable RAM in a server is probably a six figure expenditure. If memory is your constrain, this is absolutely the server for you. (sorry, should have specified that the NPU and GPU cores need to access that ram and have reasonable performance). I specified it above, but people didn't read that :-)

A basic brand new server can easily do 512gb. Not as fast as soldered memory but it should be maybe mid to high 5 figures

Re: Apple M3 Ultra

#59

Post by rsynnott » Wed, Mar 05, 2025, 2:36 PM UTC

reaperducer wrote: post #10

nottorp wrote: post #2

> support for more than half a terabyte of unified memory Soldered?

Soldered? Figure out a way to make it unified without also soldering it, and you'll be a billionaire. Or are you just grinding a tired, 20-year-old axe.

_That_, in itself, wouldn't be that difficult, and there are shared-memory setups that do use modular memory. Where you'd really run into trouble is making it _fast_; this is very, very high bandwidth memory.

Re: Apple M3 Ultra

#60

Post by jsheard » Wed, Mar 05, 2025, 2:36 PM UTC

lauritz wrote: post #46

They update the Studio to M3 Ultra now, so M4 Ultra can presumably go directly into the Mac Pro at WWDC? Interesting timing. Maybe they'll change the form factor of the Mac Pro, too? Additionally, I would assume this is a very low-volume product, so it being on N3B isn't a dealbreaker. At the same time, these chips must be very expensive to make, so tying them with luxury-priced RAM makes some kind of sense.

> Maybe they'll change the form factor of the Mac Pro, too?

Either that or kill the Mac Pro altogether, the current iteration is such a half-assed design and blatantly terrible value compared to the Studio that it feels like an end-of-the-road product just meant to tide PCIe users over until they can migrate everything to Thunderbolt.

They recycled a design meant to accommodate multiple beefy GPUs even though GPUs are no longer supported, so most of the cooling and power delivery is vestigial. Plus the PCIe expansion was quietly downgraded, Apple Silicon doesn't have a ton of PCIe lanes so the slots are heavily oversubscribed with PCIe switches.