Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

New Mac Studio with M5 Max and M5 Ultra

apple.com

Post reply on HN
Print view View original

561–570 of 570 posts

Re: New Mac Studio with M5 Max and M5 Ultra

#561

Post by angoragoats »

aurareturn wrote: post #550
angoragoats wrote: post #539

Earlier quoted context omitted.

Yes, I agree. What about the other metrics, including performance per watt which I’ve now mentioned four times and you’ve ignored the previous three times?

https://www.youtube.com/watch?v=nwIZ5VI3Eus This is a good video to watch. Yes, I agree. Glad you agree. I was just confused why you were questioning it. The big biggest advantage for Apple Silicon is that you can get much more VRAM per $ over Nvidia cards. No controversy.

You seem to be acting deliberately obtuse.

> What about [...] performance per watt which I’ve now mentioned ~~four~~ five times and you’ve ignored the previous ~~three~~ four times?

Re: New Mac Studio with M5 Max and M5 Ultra

#562

Post by seanmcdirmid »

SwellJoe wrote: post #516
seanmcdirmid wrote: post #507

Earlier quoted context omitted.

Where are you getting DeepSeek for a few bucks a month? I spend around $3/day on it, sometimes $4.

I didn't mention DeepSeek, though with the caching effectiveness in Reasonix, you can get DeepSeek for pretty danged cheap directly from deepseek.com, even at the higher prices. I was referring to the many subscriptions available that will sell you quite a lot of usage for $20/month. Given how slow local inference is, you're probably not going to ever get your money back out of it if you just use it for inference (or…

I don't think the $20/month packages come with API access at all.

I bought a refurbished M3 Max a couple of years ago just to try things out, I get 90 toks/s which is good enough for a lot of tasks. But ya, when DRAM prices come back down I'll definitely look for a better local rig.

Re: New Mac Studio with M5 Max and M5 Ultra

#563

Post by ikomrad »

nsbk wrote: post #406
joshstrange wrote: post #7

Due to pricing insanity (not that Apple prices weren’t insane before the ram/ssd shortages) I’m not in the market but I do wonder if my next computer should be a Mac Studio instead of a MBP that lives its life docked. Might be better to just run a Studio and Neo for the very few times I actually need remote capabilities.

I’m in the same camp as you. I’ve just decided that the M5 Ultra is gonna be my next machine. Plus an Air for the few times when I need a computer on the go. I’m invested in Tailscale already so an ubiquitous Studio Ultra seems like a sweet idea

I don’t even know what a Mac Ultra looks like and I want one.

It will be new Mac with more GPU cores for ai, right?

It would sit in my basement server room providing local inference.

Re: New Mac Studio with M5 Max and M5 Ultra

#564

Post by seanmcdirmid »

angoragoats wrote: post #544
seanmcdirmid wrote: post #498

Earlier quoted context omitted.

Sharding a dense model using Tensor Parallelism (TP) across dual RTX 5090s has a significantly worse performance penalty over PCIe than sharding an MoE model. If you are using multiple GPUs, MoE is basically going to be your only workable choice unless you can leverage pipeline parallelism (only half your GPUs can work on a prompt at a time, so you need to process prompts back to back in a pipeline setup, and they be…

Have you ever actually set up a multi-GPU system for inference? Based on my experience you are drastically overstating the problem. Both tensor and pipeline parallelism (without NVLink) produce a machine which is faster than any Mac on the planet, which is what we’re discussing here. Yes, each has pros and cons, and neither scales perfectly linearly. But it works great regardless.

No, and at $4000+ per 5090, I'm unlikely to have any experience anytime soon.

Re: New Mac Studio with M5 Max and M5 Ultra

#565

Post by angoragoats »

seanmcdirmid wrote: post #564
angoragoats wrote: post #544

Earlier quoted context omitted.

Have you ever actually set up a multi-GPU system for inference? Based on my experience you are drastically overstating the problem. Both tensor and pipeline parallelism (without NVLink) produce a machine which is faster than any Mac on the planet, which is what we’re discussing here. Yes, each has pros and cons, and neither scales perfectly linearly. But it works great regardless.

No, and at $4000+ per 5090, I'm unlikely to have any experience anytime soon.

You can parallelize inference with much cheaper GPUs as well!

I’ve got a 4060 ti 16gb, and I’m thinking about getting another. I previously specced out a cluster using multiple 3090s. At the time, the 3090s were going for $700 on eBay. They’re more than that now, but there’s no need to spend $4k per GPU at all.

Re: New Mac Studio with M5 Max and M5 Ultra

#566

Post by raddan »

matheusmoreira wrote: post #313
thefaux wrote: post #302

Earlier quoted context omitted.

It sounds like you don't code much in bed or with your feet up. A laptop is a distinct tool from a desktop and if you try to use it like a desktop, I agree it is a terrible substitute. Personally I find laptops vastly more ergonomic than a desktop, but I never program with my feet on the ground.

> code much in bed I've found my smartphone's the best device for that. I wrote an entire programming language using my phone. Been trying to make a handheld cyberdeck to replace the phone with something better, but it's still a long way from being real.

Can you share more about your setup? I sort of hate my laptop. I carry a ridiculously overpowered phone around with me all the time, though, so the dream is just to dock it at home or at work and just use that. It’s a shame that Apple does not make this a better experience with iPhones; I would happily shell out for the most expensive iPhone if it were a real desktop replacement. Being able to develop (for real, with a terminal) is a must though.

Re: New Mac Studio with M5 Max and M5 Ultra

#567

Post by drnick1 »

chainwax wrote: post #220
drnick1 wrote: post #204

Why doesn't Apple make a tower PC with a standard ATX motherboard, power supply, extension cards, etc, like in the old days? That's immensely more upgradable, easier to cool, and I don't believe most people expect their desktops PCs to be tiny.

I'm actually coming around to realizing that I _do_ expect a desktop PC to be small. I'm finding myself only considering Mini ITX desktop cases when I think about upgrading my desktop. I only have one video card and don't have storage requirements that necessitate lots of 3.5" drives, a big nvme is all I need. We no longer need CD/DVD/floppy drives, storage has shrunk/moved to the cloud. The only thing that's really…

> Maybe it's just me, but I think ATX full and mid towers are going the way of the dinosaur.

Even standard ATX cases are kind of small if you want to build a full custom loop and need space for radiators, pumps, reservoirs, tubes, etc. My O11 XL is admittedly enormous, but it is under my desk and does not occupy any space that wouldn't otherwise remain unused.

As for storage, I categorically refuse to store things in the cloud. My files belong at home on my hardware, not on someone else's computer in the "cloud."

Re: New Mac Studio with M5 Max and M5 Ultra

#568

Post by SwellJoe »

seanmcdirmid wrote: post #562
SwellJoe wrote: post #516

Earlier quoted context omitted.

I didn't mention DeepSeek, though with the caching effectiveness in Reasonix, you can get DeepSeek for pretty danged cheap directly from deepseek.com, even at the higher prices. I was referring to the many subscriptions available that will sell you quite a lot of usage for $20/month. Given how slow local inference is, you're probably not going to ever get your money back out of it if you just use it for inference (or…

I don't think the $20/month packages come with API access at all. I bought a refurbished M3 Max a couple of years ago just to try things out, I get 90 toks/s which is good enough for a lot of tasks. But ya, when DRAM prices come back down I'll definitely look for a better local rig.

The Qwen and Z.ai coding plans include API access, last I checked. OpenCode Go also has API access to a bunch of models for $10/month.

Re: New Mac Studio with M5 Max and M5 Ultra

#569

Post by blints »

root_axis wrote: post #388
bewareofscams wrote: post #233

Earlier quoted context omitted.

6 Mac Studios for 100k USD. Consider it a rule of thumb now - 100k to run 1T params, scales linearly.

If you have 100k you can just buy a few b200s.

One does not simply "buy a few b200". They come in 8 packs minimally, at >5x that budget.

Re: New Mac Studio with M5 Max and M5 Ultra

#570

Post by matheusmoreira »

raddan wrote: post #566
matheusmoreira wrote: post #313

Earlier quoted context omitted.

> code much in bed I've found my smartphone's the best device for that. I wrote an entire programming language using my phone. Been trying to make a handheld cyberdeck to replace the phone with something better, but it's still a long way from being real.

Can you share more about your setup? I sort of hate my laptop. I carry a ridiculously overpowered phone around with me all the time, though, so the dream is just to dock it at home or at work and just use that. It’s a shame that Apple does not make this a better experience with iPhones; I would happily shell out for the most expensive iPhone if it were a real desktop replacement. Being able to develop (for real, with…

Android phone with termux and all the usual Linux tools. I use vim to write code, and I type with the touch screen. Not exactly peak ergonomics, but it works. For me, ability to code while lying down comfortably beats having a physical keyboard and bigger screen.

Set up wireguard in my router and now I've got remote access into my LAN. My laptop has essentially become a Linux server. I just ssh in from the phone and write code. Can also easily launch and use codex.

Post reply on HN