Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

New Mac Studio with M5 Max and M5 Ultra

apple.com

Post reply on HN
Print view View original

461–470 of 570 posts

Re: New Mac Studio with M5 Max and M5 Ultra

#461

Post by seanmcdirmid »

angoragoats wrote: post #458
aurareturn wrote: post #447

Earlier quoted context omitted.

Mac studio wins in memory capacity, price, perf/watt and value. RTX 6000 wins in performance, if your model can fit into the VRAM. There are very obvious and clear advantages to a Mac Studio. It's an entire system for one and you're getting a world class CPU as well.

> Mac studio wins in memory capacity, price, perf/watt and value. [citation needed]. I have personally specced out and built an nvidia GPU-based machine which after some optimization, handily beat the Mac Studio in terms of tokens/watt for LLM inference with most models. This was in the M2 Ultra era, and I haven't run the numbers for the later generations, but nvidia's cards have gotten faster just as Apple's CPUs/GP…

> "if your model can fit into the VRAM" can be true for the Mac as well.

It is much more likely for your model to fit in large unified memory of a Mac than the smaller more limited memory of a GPU. Even going with two 5090s, you now have to shard your model and that is a PITA.

But it turns out that MoE is the solution both for running models on macs of limited computer power means (not as fast as GPUs), and on multiple GPUs that require sharding the model.

Re: New Mac Studio with M5 Max and M5 Ultra

#462

Post by aurareturn »

angoragoats wrote: post #458
aurareturn wrote: post #447

Earlier quoted context omitted.

Mac studio wins in memory capacity, price, perf/watt and value. RTX 6000 wins in performance, if your model can fit into the VRAM. There are very obvious and clear advantages to a Mac Studio. It's an entire system for one and you're getting a world class CPU as well.

> Mac studio wins in memory capacity, price, perf/watt and value. [citation needed]. I have personally specced out and built an nvidia GPU-based machine which after some optimization, handily beat the Mac Studio in terms of tokens/watt for LLM inference with most models. This was in the M2 Ultra era, and I haven't run the numbers for the later generations, but nvidia's cards have gotten faster just as Apple's CPUs/GP…

  [citation needed].
No need. You can infer the logic with this line I wrote:

  RTX 6000 wins in performance, if your model can fit into the VRAM.
I'm not sure what the controversy is here.

Re: New Mac Studio with M5 Max and M5 Ultra

#463

Post by angoragoats »

aurareturn wrote: post #462
angoragoats wrote: post #458

Earlier quoted context omitted.

> Mac studio wins in memory capacity, price, perf/watt and value. [citation needed]. I have personally specced out and built an nvidia GPU-based machine which after some optimization, handily beat the Mac Studio in terms of tokens/watt for LLM inference with most models. This was in the M2 Ultra era, and I haven't run the numbers for the later generations, but nvidia's cards have gotten faster just as Apple's CPUs/GP…

[citation needed]. No need. You can infer the logic with this line I wrote: RTX 6000 wins in performance, if your model can fit into the VRAM. I'm not sure what the controversy is here.

You made claims about performance per watt and other metrics which were completely unsubstantiated and aren't backed up by the line you quoted. That's what I was asking for citations about.

Re: New Mac Studio with M5 Max and M5 Ultra

#464

Post by dbtc »

PaulRobinson wrote: post #81
joshstrange wrote: post #7

Due to pricing insanity (not that Apple prices weren’t insane before the ram/ssd shortages) I’m not in the market but I do wonder if my next computer should be a Mac Studio instead of a MBP that lives its life docked. Might be better to just run a Studio and Neo for the very few times I actually need remote capabilities.

I've been thinking about this a fair bit recently. We make a lot of price/performance compromises for having an attached screen and keyboard on our computer. That was what got me started. Then I remembered the days of having to go to a special corner of the house to use a computer, vs now when I have a computer with me all the time. In my bag, on the sofa, on the train. Hell, I'm writing this on the work MBP while wa…

As long as we keep going back-and-forth and buying stuff, trying to figure it out, the economy will be in good shape.

Re: New Mac Studio with M5 Max and M5 Ultra

#465

Post by angoragoats »

seanmcdirmid wrote: post #461
angoragoats wrote: post #458

Earlier quoted context omitted.

> Mac studio wins in memory capacity, price, perf/watt and value. [citation needed]. I have personally specced out and built an nvidia GPU-based machine which after some optimization, handily beat the Mac Studio in terms of tokens/watt for LLM inference with most models. This was in the M2 Ultra era, and I haven't run the numbers for the later generations, but nvidia's cards have gotten faster just as Apple's CPUs/GP…

> "if your model can fit into the VRAM" can be true for the Mac as well. It is much more likely for your model to fit in large unified memory of a Mac than the smaller more limited memory of a GPU. Even going with two 5090s, you now have to shard your model and that is a PITA. But it turns out that MoE is the solution both for running models on macs of limited computer power means (not as fast as GPUs), and on multip…

> It is much more likely for your model to fit in large unified memory of a Mac than the smaller more limited memory of a GPU.

I bristle at general statements like this when it obviously depends on the specific Mac and GPU in question. But yes, comparing a maxed out M5 Ultra with an RTX 6000, the Mac has much more memory.

> Even going with two 5090s, you now have to shard your model and that is a PITA.

Every modern tool does this for you automatically. It is absolutely not a pain in the least (e.g. llama.cpp ships with pipeline parallelism enabled by default).

Re: New Mac Studio with M5 Max and M5 Ultra

#466

Post by baxtr »

abustamam wrote: post #423
loehnsberg wrote: post #404

Earlier quoted context omitted.

On some days I wor from home but need to be present and the sun is out. So I take my laptop outside and work from there. I cannot do this with a desktop. Wouldn‘t the workaholic just stay inside and miss out?

Sometimes I'll just pop into a meeting from my phone and walk around. That said , I do have a laptop but only because I go to the office every so often . Otherwise, I work exclusively from my home office because that's where my home desktop computer is and I control everything on my work computer from there.

Camera on or off?

I love taking meetings while walking but since Covid everyone has their camera on making sedentary meetings mandatory.

Re: New Mac Studio with M5 Max and M5 Ultra

#469

Post by olyjohn »

reddit_clone wrote: post #357
msdz wrote: post #248

Earlier quoted context omitted.

Yeah, I agree (with the same unfortunate habit) – but up to twice a day of forced work full stop is overdoing it, too.

Is there no hibernation option ?

Pretty sure hibernation is dead. Deep sleep states use such nominal power, I think they decided it wasn't worth it.

Re: New Mac Studio with M5 Max and M5 Ultra

#470

Post by JohnTHaller »

joshstrange wrote: post #7

Due to pricing insanity (not that Apple prices weren’t insane before the ram/ssd shortages) I’m not in the market but I do wonder if my next computer should be a Mac Studio instead of a MBP that lives its life docked. Might be better to just run a Studio and Neo for the very few times I actually need remote capabilities.

If it's your remote-in laptop, I'd suggest getting a used MacBook Pro or Air with an M series processor in your preferred 13/15" size and memory/storage configuration. You can get a used MacBook Air M3 with 16GB of RAM for about the same price as a MacBook Neo. It'll have similar single core, GPU, and NPU performance as the Neo and about 38% better multi-core performance. Plus you'll have 16GB of RAM for future-proofing and apps. And you can spend a bit less if you want or a bit more depending on what you need. You could get a used MacBook Air M1 8GB for under $400.
Post reply on HN