Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Apple M3 Ultra

apple.com

Post reply on HN
Print view View original

291–300 of 1001 posts

Re: Apple M3 Ultra

#291

Post by whimsicalism »

gatienboquet wrote: post #107
behnamoh wrote: post #35

819GB/s bandwidth... what's the point of 512GB RAM for LLMs on this Mac Studio if the speed is painfully slow? it's as if Apple doesn't want to compete with Nvidia... this is really disappointing in a Mac Studio. FYI: M2 Ultra already has 800GB/s bandwidth

NVIDIA RTX 4090: ~1,008 GB/s NVIDIA RTX 4080: ~717 GB/s AMD Radeon RX 7900 XTX: ~960 GB/s AMD Radeon RX 7900 XT: ~800 GB/s How's that slow exactly ? You can have 10000000Gb/s and without enough VRAM it's useless.

h100 sxm - 3TB/s

vram is not really the limiting factor for serious actors in this space

Re: Apple M3 Ultra

#292

Post by kjreact »

internetter wrote: post #175
rewtraw wrote: post #48

Earlier quoted context omitted.

because that's how much its worth

Its not though. For consumer computers somewhere 1k-4k there's nothing better. But for the price of 512gb of RAM you could buy that + a crazy CPU + 2x 5090s by building your own. The market fit is "needs power; needs/wants macOS; has no budget" which is so incredibly niche. But in terms of raw compute output there's absolutely no chance this is providing bang for buck

2x 5090s would only give you 64GB of memory to work with re:LLM workloads, which is what people are talking about in this thread. The 512GB of system RAM you’re referring to would not be useful in this context. Apple’s unified memory architecture is the part you’re missing.

Re: Apple M3 Ultra

#293

Post by dagmx »

llm_nerd wrote: post #195
dagmx wrote: post #163

Earlier quoted context omitted.

You're comparing two different things. The compute level you’re talking about on the M3 Ultra is the neural engine. Not including the GPU. I expect the GPU here will be behind a 5090 for compute but not by the unrelated numbers you’re quoting. After all, the 5090 alone is multiple times the wattage of this SoC.

Using the NPU numbers grossly overstates the AI performance of the Apple Silicon hardware, so they're actually giving Apple the benefit of the doubt. Most AI training and inference (including generative AI) is bound by large scale matrix MACs. That's why nvidia fills their devices with enormous numbers of tensor cores and Apple / Qualcomm et al are adding NPUs, filling largely the same gap. Only nvidia's not only are…

Care to share the TOPs numbers for the Apple GPUs and show how this would “grossly overstate” the numbers?

Apple won’t compete with NVIDIA, I’m not arguing that. But your opening line will only make sense if you can back up the numbers and the GPU performance is lower than the ANE TOPS.

Re: Apple M3 Ultra

#294

Post by segmondy »

ksec wrote: post #22

Previous model of M2 Ultra had max memory of 192GB. Or 128GB for Pro and some other M3 model, which I think is plenty for even 99.9% of professional task. They now bump it to 512GB . Along with insane price tag of $9499 for 512GB Mac Studio. I am pretty sure this is some AI Gold rush.

The question will be how it will perform. I suspect Deepseek, Llama405B demonstrated the need for larger memory. Right now folks could build an epyc system with that much ram or more to run Deepseek at about 6 tokens/sec for a fraction of that cost. However not everyone is a tinker, so there's a market for this for those that don't want to be bothered. You say "AI Gold rush" like it's a bad thing, it's not.

Re: Apple M3 Ultra

#295

Post by DrBenCarson »

internetter wrote: post #175
rewtraw wrote: post #48

Earlier quoted context omitted.

because that's how much its worth

Its not though. For consumer computers somewhere 1k-4k there's nothing better. But for the price of 512gb of RAM you could buy that + a crazy CPU + 2x 5090s by building your own. The market fit is "needs power; needs/wants macOS; has no budget" which is so incredibly niche. But in terms of raw compute output there's absolutely no chance this is providing bang for buck

How much VRAM do you get on those 2x 5090s?

How much would it cost to get up to 512gb?

Re: Apple M3 Ultra

#296

Post by kstrauser »

pmarreck wrote: post #200
InTheArena wrote: post #92

Earlier quoted context omitted.

No native docker support, no headless management options (enterprise strength), Limited QoS management, lack of robust python support (out of the box), interactive user focused security model.

> lack of robust python support There is no such thing. Tell me, which combination of the 15+ virtual environments, dependency management and Python version managers would you use? And how would you prevent "project collision" (where one Python project bumps into another one and one just stops working)? Example: SSL library differences across projects is a notorious culprit. Python is garbage and I don't understand w…

Virtualenv’s been a thing for many years, it’s built into Python, and it solves all that without adding additional tooling.

And if you’re genuinely asking, everything’s converging toward uv. If you pick only one, use that and be done with it.

Re: Apple M3 Ultra

#297

Post by klausa »

sroussey wrote: post #273
klausa wrote: post #47

Earlier quoted context omitted.

>I had read somewhere that the interposer that enabled this for the M1 chips where not available. With all my love and respect for "Apple rumors" writers; this was always "I read five blogposts about CPU design and now I'm an expert!" territory. The speculation was based on the M3 Maxes die shots not having the interposer visible, which... implies basically nothing whether that _could have_ been supported in an M3 Ul…

I’m guessing it’s not really a M3. No M3 has thunderbolt 5. This is a new chip with M3 marketing. I’d expect this from Intel, not Apple.

Baseline M4 doesn't have Thunderbolt 5 either; only the Pro/Max variants do.

The press-release even calls TB5 out: >Each Thunderbolt 5 port is supported by its own custom-designed controller directly on the chip.

Given that they're doing the same on A-series chips (A18 Pro with 10Gbps USB-C; A18 with USB 2.0); I imagine it's just relatively simple to swap the I/O blocks around and they're doing this for cost and/or product segmentation reasons.

Re: Apple M3 Ultra

#298

Post by hoppp »

seabrookmx wrote: post #271
criddell wrote: post #226

Earlier quoted context omitted.

The Asahi maintainer resigned recently. What that means for the future only time will tell. I probably wouldn't want to make a big investment in it right now.

Your wording makes it sound like it was a one-man show. Asahi has a really strong contributor base, new leadership[1], and the backing of Fedora via the Asahi Fedora Remix. While Hector resigning is a loss, I don't think it's a death knell for the project. [1]: https://asahilinux.org/2025/02/passing-the-torch/

He was the lead developer and very prominent figure. I think it probably boils down to funding the new developments.

Re: Apple M3 Ultra

#300

Post by cynicalpeace »

Can someone explain what it would take for Apple to overtake NVIDIA as the preferred solution for AI shops?

This is my understanding (probably incorrect in some places)

1. NVIDIA's big advantage is that they design the hardware (chips) and software (CUDA). But Apple also designs the hardware (chips) and software (Metal and MacOS).

2. CUDA has native support by AI libraries like PyTorch and Tensorflow, so works extra well during training and inference. It seems Metal is well supported by PyTorch, but not well supported by Tensorflow.

3. NVIDIA uses Linux rather than MacOS, making it easier in general to rack servers.

Post reply on HN