Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Apple M3 Ultra

apple.com

Post reply on HN
Print view View original

91–100 of 1001 posts

Re: Apple M3 Ultra

#91

Post by vlovich123 »

lynndotpy wrote: post #23
iambateman wrote: post #14

People who know more than me: they’re talking a lot about RAM and not much about GPU. Do you expect this will be able to handle AI workloads well? All I’ve heard for the past two years is how important a beefy GPU is. Curious if that holds true here too.

VRAM is what takes a model from "can not run at all" to "can run" (even if slowly), hence the emphasis.

No, with limited VRAM you could offload the model partially or split across CPU and GPU. And since CPU has swap, you could run the absolute largest model. It’s just really really slow.

Re: Apple M3 Ultra

#92

Post by InTheArena »

jagged-chisel wrote: post #79
InTheArena wrote: post #29

Whoa. M3 instead of M4. I wonder if this was basically binning, but I thought that I had read somewhere that the interposer that enabled this for the M1 chips where not available. That Said, 512GB of unified ram with access to the NPU is absolutely a game changer. My guess is that Apple developed this chip for their internal AI efforts, and are now at the point where they are releasing it publicly for others to use.…

> This hardware is really being held back by the operating system at this point. Please elucidate.

No native docker support, no headless management options (enterprise strength), Limited QoS management, lack of robust python support (out of the box), interactive user focused security model.

Re: Apple M3 Ultra

#93

Post by jeffbee »

InTheArena wrote: post #45
TheTxT wrote: post #8

512GB unified memory is absolutely wild for AI stuff! Compared to how many NVIDIA GPUs you would need, the pricing looks almost reasonable.

A server with 512GB of high-bandwidth GPU addressable RAM in a server is probably a six figure expenditure. If memory is your constrain, this is absolutely the server for you. (sorry, should have specified that the NPU and GPU cores need to access that ram and have reasonable performance). I specified it above, but people didn't read that :-)

That doesn't sound right. The marginal cost of +768GB of DDR5 ECC memory in an EPYC system is < $5k.

Re: Apple M3 Ultra

#94

Post by [deleted] »

diggan wrote: post #90
jagged-chisel wrote: post #79

Earlier quoted context omitted.

> This hardware is really being held back by the operating system at this point. Please elucidate.

https://news.ycombinator.com/item?id=43243075 ("Apple's Software Quality Crisis" - 1134 comments) ^ has a lot of elaborations on this subject

[deleted]

Re: Apple M3 Ultra

#95

Post by mythz »

Shank wrote: post #69
moondev wrote: post #65

> support for more than half a terabyte of unified memory — the most ever in a personal computer AMD Ryzen Threadripper PRO 3995WX released over four years ago and supports 2TB (64c/128t) > Take your workstation's performance to the next level with the AMD Ryzen Threadripper PRO 3995WX 2.7 GHz 64-Core sWRX8 Processor. Built using the 7nm Zen Core architecture with the sWRX8 socket, this processor is designed to deliv…

> unified memory So unified memory means that the memory is accessible to the GPU and the CPU in a shared pool. AMD does not have that.

AMD Ryzen AI Max SoC chips have that [1], but it maxes out at 128GB RAM.

[1] https://www.amd.com/en/products/processors/laptop/ryzen/ai-3...

Re: Apple M3 Ultra

#96

Post by [deleted] »

InTheArena wrote: post #45
TheTxT wrote: post #8

512GB unified memory is absolutely wild for AI stuff! Compared to how many NVIDIA GPUs you would need, the pricing looks almost reasonable.

A server with 512GB of high-bandwidth GPU addressable RAM in a server is probably a six figure expenditure. If memory is your constrain, this is absolutely the server for you. (sorry, should have specified that the NPU and GPU cores need to access that ram and have reasonable performance). I specified it above, but people didn't read that :-)

[deleted]

Re: Apple M3 Ultra

#97

Post by InTheArena »

la_oveja wrote: post #80
Numerlor wrote: post #58

Earlier quoted context omitted.

A basic brand new server can easily do 512gb. Not as fast as soldered memory but it should be maybe mid to high 5 figures

5 figures? can be done in 6k https://x.com/carrigmat/status/1884244369907278106

That's CPU only memory, not high bandwidth, and not addressable by the GPU.

Re: Apple M3 Ultra

#98

Post by behnamoh »

mrtksn wrote: post #61

Let's say you want to have the absolute max memory(512GB) to run AI models and let's say that you are O.K. with plugging a drive to archive your model weights then you can get this for a little bit shy of $10K. What a dream machine. Compared to Nvidia's Project DIGITS which is supposed to cost $3K and be available "soon", you can get a specs matching 128GB & 4TB version of this Mac for about $4700 and the difference…

> I can't wait to see someone testing the full DeepSeek model on this

at 819 GB per second bandwidth, the experience would be terrible

Re: Apple M3 Ultra

#99

Post by layer8 »

behnamoh wrote: post #83
InTheArena wrote: post #29

Whoa. M3 instead of M4. I wonder if this was basically binning, but I thought that I had read somewhere that the interposer that enabled this for the M1 chips where not available. That Said, 512GB of unified ram with access to the NPU is absolutely a game changer. My guess is that Apple developed this chip for their internal AI efforts, and are now at the point where they are releasing it publicly for others to use.…

> My guess is that Apple developed this chip for their internal AI efforts what internal AI efforts? Apple Intelligence is bunkers, and Apple MLX framework remains a hobby project for Apple

https://security.apple.com/blog/private-cloud-compute/

https://security.apple.com/documentation/private-cloud-compu...

https://techcrunch.com/2024/12/11/apple-reportedly-developin...

https://techcrunch.com/2025/02/24/apple-commits-500b-to-us-m...

Re: Apple M3 Ultra

#100

Post by InTheArena »

jeffbee wrote: post #93
InTheArena wrote: post #45

Earlier quoted context omitted.

A server with 512GB of high-bandwidth GPU addressable RAM in a server is probably a six figure expenditure. If memory is your constrain, this is absolutely the server for you. (sorry, should have specified that the NPU and GPU cores need to access that ram and have reasonable performance). I specified it above, but people didn't read that :-)

That doesn't sound right. The marginal cost of +768GB of DDR5 ECC memory in an EPYC system is < $5k.

GPU accessible RAM.
Post reply on HN