Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

macOS 26.2 enables fast AI clusters with RDMA over Thunderbolt

developer.apple.com

Post reply on HN
Print view View original

121–130 of 304 posts

Re: macOS 26.2 enables fast AI clusters with RDMA over Thunderbolt

#121

Post by [deleted] »

reilly3000 wrote: post #105
3abiton wrote: post #89

Earlier quoted context omitted.

What's the math on the $50k nvidia cluster? My understanding these things cost ~$8k and you can at least get 5 for $40k, that's around half a tb. That being said, for inference mac still remain the best, and the M5 Ultra will even be a better value with its better PP.

GPUs: 4x NVIDIA RTX 6000 Blackwell (96GB VRAM each) • Cost: 4 × $9,000 = $36,000 • CPU: AMD Ryzen Threadripper PRO 7995WX (96-Core) • Cost: $10,000 • Motherboard: WRX90 Chipset (supports 7x PCIe Gen5 slots) • Cost: $1,200 • RAM: 512GB DDR5 ECC Registered • Cost: $2,000 • Chassis & Power: Supermicro or specialized Workstation case + 2x 1600W PSUs. • Cost: $1,500 • Total Cost: ~$50,700 It’s a bit maximalist, but if you…

[deleted]

Re: macOS 26.2 enables fast AI clusters with RDMA over Thunderbolt

#122

Post by jabbywocker »

firecall wrote: post #116
jabbywocker wrote: post #111

Earlier quoted context omitted.

I’m expecting Apple to release a new Mac Pro in the next couple years who’s main marketing angle is exactly this

Seems like it could be a thing. Also, I’m curious and in case anyone that knows reads this comment: Apple say they can’t get the performance they want out of discreet GPUs. Fair enough. But yet nVidia becomes the most valuable company in the world selling GPUs. So… Now I get that Apples use case is essentially sealed consumer devices built with power consumption and performance tradeoffs in mind. But could Apple use…

There’s been rumors of Apple working on M-chips that have the GPU and CPU as discrete chiplets. The original rumor said this would happen with the M5 Pro, so it’s potentially on the roadmap.

Theoretically they could farm out the GPU to another company but it seems like they’re set on owning all of the hardware designs.

Re: macOS 26.2 enables fast AI clusters with RDMA over Thunderbolt

#123

Post by moralestapia »

londons_explore wrote: post #86
moralestapia wrote: post #82

Earlier quoted context omitted.

You might ignore this but, for a while, Mac Mini clusters were a thing and they were capex and opex effective. That same setup is kind of making a comeback.

It's in a similar vein to the PS2 linux cluster or someone trying to use vape CPU's as web servers... It might be cost effective, but the supplier is still saying "you get no support, and in fact we might even put roadblocks in your way because you aren't the target customer".

True.

I'm sure Apple could make a killing on the server side, unfortunately their income from their other products is so big that even if that's a 10B/year opportunity they'll be like "yawn, yeah, whatever".

Re: macOS 26.2 enables fast AI clusters with RDMA over Thunderbolt

#124

Post by pests »

bee_rider wrote: post #104
sspiff wrote: post #87

Earlier quoted context omitted.

There is no way to achieve a high throughput low latency connection between 25 Strix Halo systems. After accounting for storage and network, there are barely any PCIe lanes left to link two of them together. You might be able to use USB4 but unsure how the latency is for that.

Do they need fast storage, in this application? Their OS could be on some old SATA drive or whatever. The whole goal is to get them on a fast network together; the models could be stored on some network filesystem as well, right?

It's more than just the model weights. During inference there would be a lot of cross-talk as each node broadcasts its results and gathers up what it needs from the others for the next step.

Re: macOS 26.2 enables fast AI clusters with RDMA over Thunderbolt

#125

Post by mirekrusin »

icedchai wrote: post #70
andy99 wrote: post #11

Earlier quoted context omitted.

It’s gonna suck if all the good Macs get gobbled up by commercial users.

Outside of YouTube influencers, I doubt many home users are buying a 512G RAM Mac Studio.

Of course they're not. Everybody is waiting for next generation that will run LLMs faster to start buying.

Re: macOS 26.2 enables fast AI clusters with RDMA over Thunderbolt

#126

Post by pests »

behnamoh wrote: post #119
dghlsakjg wrote: post #93

Earlier quoted context omitted.

Huh? Home PCs are as cheap as they’ve ever been. Adjusted for inflation the same can be said about “home use” Macs. The list price of an entry level MacBook Air has been pretty much the same for more than a decade. Adjust for inflation, and you get a MacBook air for less than half the real cost of the launch model that is massively better in every way. A blip in high end RAM prices has no bearing on affordable home c…

> Home PCs are as cheap as they’ve ever been. just the 5090 GPU costs +$3k, what are you even talking about

A home PC has to have a SOTA gpu?

Re: macOS 26.2 enables fast AI clusters with RDMA over Thunderbolt

#127

Post by Weryj »

mechagodzilla wrote: post #55
reilly3000 wrote: post #48

dang I wish I could share md tables. Here’s a text edition: For $50k the inference hardware market forces a trade-off between capacity and throughput: * Apple M3 Ultra Cluster ($50k): Maximizes capacity (3TB). It is the only option in this price class capable of running 3T+ parameter models (e.g., Kimi k2), albeit at low speeds (~15 t/s). * NVIDIA RTX 6000 Workstation ($50k): Maximizes throughput (>80 t/s). It is sup…

You can keep scaling down! I spent $2k on an old dual-socket xeon workstation with 768GB of RAM - I can run Deepseek-R1 at ~1-2 tokens/sec.

Just keep going! 2TB of swap disk for 0.0000001 t/sec

Re: macOS 26.2 enables fast AI clusters with RDMA over Thunderbolt

#128

Post by dghlsakjg »

behnamoh wrote: post #119
dghlsakjg wrote: post #93

Earlier quoted context omitted.

Huh? Home PCs are as cheap as they’ve ever been. Adjusted for inflation the same can be said about “home use” Macs. The list price of an entry level MacBook Air has been pretty much the same for more than a decade. Adjust for inflation, and you get a MacBook air for less than half the real cost of the launch model that is massively better in every way. A blip in high end RAM prices has no bearing on affordable home c…

> Home PCs are as cheap as they’ve ever been. just the 5090 GPU costs +$3k, what are you even talking about

“A computer in every home” (from the original post I was replying to) does not mean “A computer with the highest priced version of the highest priced optional accessory for computers in every home”

I’m talking about the hundreds of affordable models that are perfectly suitable for everything up to and including AAA gaming.

The existence of expensive, and very much optional, high end computer parts does not mean that affordable computers are not more incredible than ever.

Just because cutting edge high end parts are out of reach to you, does not mean that perfectly usable computers are too, as I demonstrated with actual specs and prices in my post.

That’s what I’m talking about.

Re: macOS 26.2 enables fast AI clusters with RDMA over Thunderbolt

#130

Post by a012 »

mechagodzilla wrote: post #55
reilly3000 wrote: post #48

dang I wish I could share md tables. Here’s a text edition: For $50k the inference hardware market forces a trade-off between capacity and throughput: * Apple M3 Ultra Cluster ($50k): Maximizes capacity (3TB). It is the only option in this price class capable of running 3T+ parameter models (e.g., Kimi k2), albeit at low speeds (~15 t/s). * NVIDIA RTX 6000 Workstation ($50k): Maximizes throughput (>80 t/s). It is sup…

You can keep scaling down! I spent $2k on an old dual-socket xeon workstation with 768GB of RAM - I can run Deepseek-R1 at ~1-2 tokens/sec.

And heat the whole house in parallel
Post reply on HN