Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

macOS 26.2 enables fast AI clusters with RDMA over Thunderbolt

developer.apple.com

Post reply on HN
Print view View original

141–150 of 304 posts

Re: macOS 26.2 enables fast AI clusters with RDMA over Thunderbolt

#141

Post by platevoltage »

behnamoh wrote: post #119
dghlsakjg wrote: post #93

Earlier quoted context omitted.

Huh? Home PCs are as cheap as they’ve ever been. Adjusted for inflation the same can be said about “home use” Macs. The list price of an entry level MacBook Air has been pretty much the same for more than a decade. Adjust for inflation, and you get a MacBook air for less than half the real cost of the launch model that is massively better in every way. A blip in high end RAM prices has no bearing on affordable home c…

> Home PCs are as cheap as they’ve ever been. just the 5090 GPU costs +$3k, what are you even talking about

Man you positively demolished that straw man.

How much as a base model MacBook Air changed in price over the last 15 years? With inflation, it's gotten cheaper.

Re: macOS 26.2 enables fast AI clusters with RDMA over Thunderbolt

#142

Post by platevoltage »

heavyset_go wrote: post #100
dghlsakjg wrote: post #93

Earlier quoted context omitted.

Huh? Home PCs are as cheap as they’ve ever been. Adjusted for inflation the same can be said about “home use” Macs. The list price of an entry level MacBook Air has been pretty much the same for more than a decade. Adjust for inflation, and you get a MacBook air for less than half the real cost of the launch model that is massively better in every way. A blip in high end RAM prices has no bearing on affordable home c…

Home calculators are cheap as they've ever been, but this era of computing is out of reach for the majority of people. The analogous PC for this era requires a large amount of high speed memory and specialized inference hardware.

No it doesn't. The majority of people aren't trying to run Ollama on their personal computers.

Re: macOS 26.2 enables fast AI clusters with RDMA over Thunderbolt

#143

Post by bleepblap »

kmeisthax wrote: post #91
huslage wrote: post #34

Earlier quoted context omitted.

I don't believe the standard supports such a thing. But I wonder if TB6 will.

RDMA is a networking standard, it's supposed to be switched. The reason why it's being done over Thunderbolt is that it's the only cheap/prosumer I/O standard with enough bandwidth to make this work. Like, 100Gbit Ethernet cards are several hundred dollars minimum, for two ports, and you have to deal with SFP+ cabling. Thunderbolt is just way nicer[0]. The way this capability is exposed in the OS is that the computer…

I think you might be swapping RDMA with RoCE - RDMA can happen entirely within a single node. For example between an NVME and a GPU.

Re: macOS 26.2 enables fast AI clusters with RDMA over Thunderbolt

#144

Post by rsync »

geerlingguy wrote: post #21

This implies you'd run more than one Mac Studio in a cluster, and I have a few concerns regarding Mac clustering (as someone who's managed a number of tiny clusters, with various hardware): 1. The power button is in an awkward location, meaning rackmounting them (either 10" or 19" rack) is a bit cumbersome (at best) 2. Thunderbolt is great for peripherals, but as a semi-permanent interconnect, I have worries over the…

"... Thunderbolt is great for peripherals, but as a semi-permanent interconnect, I have worries over the port's physical stability ..."

Thunderbolt as a server interconnect displeases me aesthetically but my conclusion is the opposite of yours:

If the systems are locked into place as servers in a rack the movements and stresses on the cable are much lower than when it is used as a peripheral interconnect for a desktop or laptop, yes ?

Re: macOS 26.2 enables fast AI clusters with RDMA over Thunderbolt

#145

Post by dghlsakjg »

platevoltage wrote: post #141
behnamoh wrote: post #119

Earlier quoted context omitted.

> Home PCs are as cheap as they’ve ever been. just the 5090 GPU costs +$3k, what are you even talking about

Man you positively demolished that straw man. How much as a base model MacBook Air changed in price over the last 15 years? With inflation, it's gotten cheaper.

Some numbers to drive your point home:

The original base MacBook Air sold for $1799 in 2008. The inflation adjusted price is $2715.

The current base model is $999, and literally better in every way except thickness on one edge.

If we constrain ourselves to just 15 years. The $999 MBA was released 15 years ago ($1488 in real dollars). The list price has remained the same for the base model, with the exception of when they sold the discontinued 11” MBAs for $899.

It’s actually kind of wild how much better and cheaper computers have gotten.

Re: macOS 26.2 enables fast AI clusters with RDMA over Thunderbolt

#146

Post by wmf »

dpe82 wrote: post #108
awnihannun wrote: post #29

Earlier quoted context omitted.

For a bit more context, those posts are using pipeline parallelism. For N machines put the first L/N layers on machine 1, next L/N layers on machine 2, etc. With pipeline parallelism you don't get a speedup over one machine - it just buys you the ability to use larger models than you can fit on a single machine. The release in Tahoe 26.2 will enable us to do fast tensor parallelism in MLX. Each layer of the model is…

> The main challenge is latency since you have to do much more frequent communication. Earlier this year I experimented with building a cluster to do tensor parallelism across large cache CPUs (AMD EPYC 7773X have 768mb of L3). My thought was to keep an entire model in SRAM and take advantage of the crazy memory bandwidth between CPU cores and their cache, and use Infiniband between nodes for the scatter/gather opera…

That's how Groq works. A cluster of LPUv2s would probably be faster and cheaper than an Infiniband cluster of Epycs.

Re: macOS 26.2 enables fast AI clusters with RDMA over Thunderbolt

#147

Post by badc0ffee »

geerlingguy wrote: post #21

This implies you'd run more than one Mac Studio in a cluster, and I have a few concerns regarding Mac clustering (as someone who's managed a number of tiny clusters, with various hardware): 1. The power button is in an awkward location, meaning rackmounting them (either 10" or 19" rack) is a bit cumbersome (at best) 2. Thunderbolt is great for peripherals, but as a semi-permanent interconnect, I have worries over the…

> To that last point, I've been trying to figure out a way to, for example, upgrade to macOS 26.2 from 26.1 remotely,

I think you can do this if you install a MDM profile on the Macs and use some kind of management software like Jamf.

Re: macOS 26.2 enables fast AI clusters with RDMA over Thunderbolt

#150

Post by 827a »

geerlingguy wrote: post #21

This implies you'd run more than one Mac Studio in a cluster, and I have a few concerns regarding Mac clustering (as someone who's managed a number of tiny clusters, with various hardware): 1. The power button is in an awkward location, meaning rackmounting them (either 10" or 19" rack) is a bit cumbersome (at best) 2. Thunderbolt is great for peripherals, but as a semi-permanent interconnect, I have worries over the…

They do still sell the Mac Pro in a rack mount configuration. But, it was never updated for M3 Ultra, and feels not long for this world.
Post reply on HN