Call me a unit fundamentalist but calling 512Gb "over half a terabyte memory" irks me to no end.
Perhaps they’re including the CPU cache and rounding down for brevity.
Apple M3 Ultra
21–30 of 1001 posts
Re: Apple M3 Ultra
#22Previous model of M2 Ultra had max memory of 192GB. Or 128GB for Pro and some other M3 model, which I think is plenty for even 99.9% of professional task.
They now bump it to 512GB. Along with insane price tag of $9499 for 512GB Mac Studio. I am pretty sure this is some AI Gold rush.
Re: Apple M3 Ultra
#23People who know more than me: they’re talking a lot about RAM and not much about GPU. Do you expect this will be able to handle AI workloads well? All I’ve heard for the past two years is how important a beefy GPU is. Curious if that holds true here too.
VRAM is what takes a model from "can not run at all" to "can run" (even if slowly), hence the emphasis.
Re: Apple M3 Ultra
#24People who know more than me: they’re talking a lot about RAM and not much about GPU. Do you expect this will be able to handle AI workloads well? All I’ve heard for the past two years is how important a beefy GPU is. Curious if that holds true here too.
When it comes to LLMs in particular, it comes down to memory size+bandwidth more than anything else.
Re: Apple M3 Ultra
#25Time to upgrade m1 ultra I guess! M1 ultra has been pretty good with deepseek locally.
Re: Apple M3 Ultra
#26People who know more than me: they’re talking a lot about RAM and not much about GPU. Do you expect this will be able to handle AI workloads well? All I’ve heard for the past two years is how important a beefy GPU is. Curious if that holds true here too.
What's more important isn't how beefy it is, it's how much memory it has.
These are unified memory. The M3 Ultra with 512gb has as much VRAM as sixteen 5090.
Re: Apple M3 Ultra
#27Unclear what devices this will be in outside of the mac studio. Also most of the comparisons were with M1 and M2 chips, not M4.
It is a bit misleading to do that, but in fairness to Apple, almost nobody is upgrading to this from an M4 Mac, so those are probably more useful comparisons.
Re: Apple M3 Ultra
#28> support for more than half a terabyte of unified memory Soldered?
Not even Framework has escaped from soldered RAM for this kind of thing.
Re: Apple M3 Ultra
#29Whoa. M3 instead of M4. I wonder if this was basically binning, but I thought that I had read somewhere that the interposer that enabled this for the M1 chips where not available.
That Said, 512GB of unified ram with access to the NPU is absolutely a game changer. My guess is that Apple developed this chip for their internal AI efforts, and are now at the point where they are releasing it publicly for others to use. They really need a 2U rack form for this though.
This hardware is really being held back by the operating system at this point.
Re: Apple M3 Ultra
#30Earlier quoted context omitted.
Is there a single Apple SoC where they’ve provided removable ram? Not that I can recall.
Is there even an existing replaceable memory standard that would meet the current needs of Apple's "Unified Memory" architecture? I'm not an expert but I'd suspect probably not. The bus probably looks a lot more like VRAM on GPUs, and I've never seen a GPU with replaceable RAM.
CAMM2 could kinda work, but each module is only 128-bit so I think the furthest you could possibly push it is a 512-bit M Max equivalent with CAMM2 modules north, east, west and south of the SOC. There just isn't room to put eight modules right next to the SOC for a 1024-bit bus like the M Ultra.