Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

macOS 26.2 enables fast AI clusters with RDMA over Thunderbolt

developer.apple.com

Post reply on HN
Print view View original

151–160 of 304 posts

Re: macOS 26.2 enables fast AI clusters with RDMA over Thunderbolt

#151

Post by wmf »

bleepblap wrote: post #143
kmeisthax wrote: post #91

Earlier quoted context omitted.

RDMA is a networking standard, it's supposed to be switched. The reason why it's being done over Thunderbolt is that it's the only cheap/prosumer I/O standard with enough bandwidth to make this work. Like, 100Gbit Ethernet cards are several hundred dollars minimum, for two ports, and you have to deal with SFP+ cabling. Thunderbolt is just way nicer[0]. The way this capability is exposed in the OS is that the computer…

I think you might be swapping RDMA with RoCE - RDMA can happen entirely within a single node. For example between an NVME and a GPU.

Within a single node it's just called DMA. RDMA is DMA over a network and RoCE is RDMA over Ethernet.

Re: macOS 26.2 enables fast AI clusters with RDMA over Thunderbolt

#152

Post by 827a »

rsync wrote: post #144
geerlingguy wrote: post #21

This implies you'd run more than one Mac Studio in a cluster, and I have a few concerns regarding Mac clustering (as someone who's managed a number of tiny clusters, with various hardware): 1. The power button is in an awkward location, meaning rackmounting them (either 10" or 19" rack) is a bit cumbersome (at best) 2. Thunderbolt is great for peripherals, but as a semi-permanent interconnect, I have worries over the…

"... Thunderbolt is great for peripherals, but as a semi-permanent interconnect, I have worries over the port's physical stability ..." Thunderbolt as a server interconnect displeases me aesthetically but my conclusion is the opposite of yours: If the systems are locked into place as servers in a rack the movements and stresses on the cable are much lower than when it is used as a peripheral interconnect for a deskto…

This is a semi-solved problem e.g. https://www.sonnetstore.com/products/thunderlok-a

Apple’s chassis do not support it. But conceptually that’s not a Thunderbolt problem, it’s an Apple problem. You could probably drill into the Mac Studio chassis to create mount points.

Re: macOS 26.2 enables fast AI clusters with RDMA over Thunderbolt

#154

Post by CamperBob2 »

simonw wrote: post #3

I follow the MLX team on Twitter and they sometimes post about using MLX on two or more joined together Macs to run models that need more than 512GB of RAM. A couple of examples: Kimi K2 Thinking (1 trillion parameters): https://x.com/awnihannun/status/1986601104130646266 DeepSeek R1 (671B): https://x.com/awnihannun/status/1881915166922863045 - that one came with setup instructions in a Gist: https://gist.github.com/…

Almost the most impressive thing about that is the power consumption. ~50 watts for both of them? Am I reading it wrong?

Re: macOS 26.2 enables fast AI clusters with RDMA over Thunderbolt

#155

Post by _zoltan_ »

wtallis wrote: post #114
_zoltan_ wrote: post #106

Earlier quoted context omitted.

the GH/GB compute has LPDDR5X - a single or dual GPU shares 480GB, depending if it's GH or GB, in addition to the HBM memory, with NVLink C2C - it's not bad!

Essentially, the Grace CPU is a memory and IO expander that happens to have a bunch of ARM CPU cores filling in the interior of the die, while the perimeter is all PHYs for LPDDR5 and NVLink and PCIe.

fully agree!

with MGX and CX8 we see PCIe root moving to the NIC, which is very exciting.

Re: macOS 26.2 enables fast AI clusters with RDMA over Thunderbolt

#156

Post by 7e »

bigyabai wrote: post #115
Eggpants wrote: post #103

Earlier quoted context omitted.

Not sure I understand, Mac OS is BSD based. https://en.wikipedia.org/wiki/Darwin_(operating_system)

macOS is XNU-based. There is BSD code that runs in the microkernel level and BSD tools in the userland, but the kernel does not resemble BSD's architecture or adopt BSD's license. This is an issue for some industry-standard software like CUDA, which does provide BSD drivers with ARM support that just never get adopted by Apple: https://www.nvidia.com/en-us/drivers/unix/

If there were TCO advantages with this setup, CUDA would not be a blocker.

Re: macOS 26.2 enables fast AI clusters with RDMA over Thunderbolt

#157

Post by 7e »

icedchai wrote: post #70
andy99 wrote: post #11

Earlier quoted context omitted.

It’s gonna suck if all the good Macs get gobbled up by commercial users.

Outside of YouTube influencers, I doubt many home users are buying a 512G RAM Mac Studio.

That product can still steal fab slots from cheaper, more prosumer products.

Re: macOS 26.2 enables fast AI clusters with RDMA over Thunderbolt

#158

Post by inferiorhuman »

dghlsakjg wrote: post #132
inferiorhuman wrote: post #102

Earlier quoted context omitted.

A blip in high end RAM prices It's not a blip and it's not limited to high end machines and configurations. Altman gobbled up the lion's share of wafer production. Look at that Raspberry Pi article that made it to the front page, that's pretty far from a high end Mac and according to the article's author likely to be exported from China due to the RAM supply crisis. I can get a Ryzen 7 system with 32gb of ddr5, and a…

People that can reliably predict the future, especially when it comes to rising markets, are almost always billionaires. It is a skill so rare that it can literally make you the richest man on earth. Why should I trust your prediction of future markets that this pricing is the new standard, and will never go down? Line doesn’t always go up, even if it feels like it is right now, and all the tech media darlings are sa…

  People that can reliably predict the future
You don't need to be a genius or a billionaire to realize that when most of the global supply of a product becomes unavailable the remaining supply gets more expensive.

  here’s an equivalent speced pc available in the US for $439 with a prime membership.
So with prime that's $439+139 for $578 which is only slightly higher than the cost without prime of $549.99.

Re: macOS 26.2 enables fast AI clusters with RDMA over Thunderbolt

#159

Post by morshu9001 »

platevoltage wrote: post #141
behnamoh wrote: post #119

Earlier quoted context omitted.

> Home PCs are as cheap as they’ve ever been. just the 5090 GPU costs +$3k, what are you even talking about

Man you positively demolished that straw man. How much as a base model MacBook Air changed in price over the last 15 years? With inflation, it's gotten cheaper.

It's also gotten cheaper nominally. I just got a new base MBA for $750. Kinda surprised, like there has to be some catch.

Re: macOS 26.2 enables fast AI clusters with RDMA over Thunderbolt

#160

Post by willtemperley »

andy99 wrote: post #30
simonw wrote: post #3

I follow the MLX team on Twitter and they sometimes post about using MLX on two or more joined together Macs to run models that need more than 512GB of RAM. A couple of examples: Kimi K2 Thinking (1 trillion parameters): https://x.com/awnihannun/status/1986601104130646266 DeepSeek R1 (671B): https://x.com/awnihannun/status/1881915166922863045 - that one came with setup instructions in a Gist: https://gist.github.com/…

I’m hoping this isn’t as attractive as it sounds for non-hobbyists because the performance won’t scale well to parallel workloads or even context processing, where parallelism can be better used. Hopefully this makes it really nice for people that want the experiment with LLMs and have a local model but means well funded companies won’t have any reason to grab them all vs GPUs.

I think it’s going to be great for smaller shops that want on premise private cloud. I’m hoping this will be a win for in-memory analytics on macOS.
Post reply on HN