Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

macOS 26.2 enables fast AI clusters with RDMA over Thunderbolt

developer.apple.com

Post reply on HN
Print view View original

131–140 of 304 posts

Re: macOS 26.2 enables fast AI clusters with RDMA over Thunderbolt

#131

Post by fennecbutt »

moralestapia wrote: post #82
londons_explore wrote: post #80

Nobodies gonna take them seriously till they make something rack mounted and that isn't made of titanium with pentalobe screws...

You might ignore this but, for a while, Mac Mini clusters were a thing and they were capex and opex effective. That same setup is kind of making a comeback.

They were only a thing to do ci/compilation related to apples os because their walled garden locked using other platforms out. You're building an iPhone or mac app? Well your ci needs to be on a cluster of apple machines.

Re: macOS 26.2 enables fast AI clusters with RDMA over Thunderbolt

#132

Post by dghlsakjg »

inferiorhuman wrote: post #102
dghlsakjg wrote: post #93

Earlier quoted context omitted.

Huh? Home PCs are as cheap as they’ve ever been. Adjusted for inflation the same can be said about “home use” Macs. The list price of an entry level MacBook Air has been pretty much the same for more than a decade. Adjust for inflation, and you get a MacBook air for less than half the real cost of the launch model that is massively better in every way. A blip in high end RAM prices has no bearing on affordable home c…

A blip in high end RAM prices It's not a blip and it's not limited to high end machines and configurations. Altman gobbled up the lion's share of wafer production. Look at that Raspberry Pi article that made it to the front page, that's pretty far from a high end Mac and according to the article's author likely to be exported from China due to the RAM supply crisis. I can get a Ryzen 7 system with 32gb of ddr5, and a…

People that can reliably predict the future, especially when it comes to rising markets, are almost always billionaires. It is a skill so rare that it can literally make you the richest man on earth. Why should I trust your prediction of future markets that this pricing is the new standard, and will never go down? Line doesn’t always go up, even if it feels like it is right now, and all the tech media darlings are saying so.

If everything remains the same, RAM pricing will also. I have never once found a period in known history where everything stays the same, and I would be willing to bet 5 figures that at some point in the future I will be able to buy DDR5 or better ram for cheaper than today. I can point out that in the long run, prices for computing equipment have always fallen. I would trust that trend a lot more than a shortage a few months old changing the very nature of commodity markets. Mind you, I’m not the richest man on earth either, so my pattern matched opinion should be judged the same.

> B&H is showing a 7700X at $250 with their cheapest 32GB DDR5 5200 sticks at $384. So you've already gone over budget for just the memory and CPU. No motherboard, no SSD.

I didn't say I could build one from parts. Instead I said buy a mini pc, and then went and looked up the specs and price point to be sure.

The PC that I was talking about is here[https://a.co/d/6c8Udbp]. I live in Canada so translated the prices to USD. Remember that US stores are sometimes forced to hide a massive import tax in those parts prices. The rest of the world isn’t subject to that and pays less.

Edit: here’s an equivalent speced pc available in the US for $439 with a prime membership. So even with the cost of prime membership you can get a Ryzen 7 32gb 1tb for $455. https://www.amazon.com/BOSGAME-P3-Gigabit-Ethernet-Computer/...

Re: macOS 26.2 enables fast AI clusters with RDMA over Thunderbolt

#133

Post by fennecbutt »

moralestapia wrote: post #123
londons_explore wrote: post #86

Earlier quoted context omitted.

It's in a similar vein to the PS2 linux cluster or someone trying to use vape CPU's as web servers... It might be cost effective, but the supplier is still saying "you get no support, and in fact we might even put roadblocks in your way because you aren't the target customer".

True. I'm sure Apple could make a killing on the server side, unfortunately their income from their other products is so big that even if that's a 10B/year opportunity they'll be like "yawn, yeah, whatever".

Doubt. A 10B idea is still a promotion. And if capitalism is shrinkflationing hard, which it is atm, then capitalists would not leave something like that on the table.

Re: macOS 26.2 enables fast AI clusters with RDMA over Thunderbolt

#134

Post by liuliu »

awnihannun wrote: post #112
liuliu wrote: post #71

Earlier quoted context omitted.

I usually call it "head parallelism" (which is a type of tensor parallelism, but paralllelize for small clusters, and specific to attention). That is what you described: sharding input tensor by number of heads and send to respective Q, K, V shard. They can do Q / K / V projection, rope, qk norm whatever and attention all inside that particular shard. The out projection will be done in that shard too but then need to…

Right, my comment was mostly about decoding speed. For prefill you can get a speed up but there you are less latency bound. In our benchmarks with MLX / mlx-lm it's as much as 3.5x for token generation (decoding) at batch size 1 over 4 machines. In that case you are memory bandwidth bound so sharding the model and KV cache 4-ways means each machine only needs to access 1/4th as much memory.

Oh! That's great to hear. Congrats! Now, I want to get the all-to-all primitives ready in s4nnc...

Re: macOS 26.2 enables fast AI clusters with RDMA over Thunderbolt

#135

Post by atonse »

heavyset_go wrote: post #100
dghlsakjg wrote: post #93

Earlier quoted context omitted.

Huh? Home PCs are as cheap as they’ve ever been. Adjusted for inflation the same can be said about “home use” Macs. The list price of an entry level MacBook Air has been pretty much the same for more than a decade. Adjust for inflation, and you get a MacBook air for less than half the real cost of the launch model that is massively better in every way. A blip in high end RAM prices has no bearing on affordable home c…

Home calculators are cheap as they've ever been, but this era of computing is out of reach for the majority of people. The analogous PC for this era requires a large amount of high speed memory and specialized inference hardware.

You can have access to a supercomputer for pennies, internet access for very little money, and even an m4 Mac mini for $500. You can have a raspberry pi computer for even less. And buy a monitor for a couple hundred dollars.

I feel like you’re twisting the goalposts to make your point that it has to be local compute to have access to AI. Why does it need to be local?

Update: I take it back. You can get access to AI for free.

Re: macOS 26.2 enables fast AI clusters with RDMA over Thunderbolt

#136

Post by Waterluvian »

DrStartup wrote: post #118
icedchai wrote: post #70

Earlier quoted context omitted.

Outside of YouTube influencers, I doubt many home users are buying a 512G RAM Mac Studio.

I'm neither and have 2. 24/7 async inference against github issues. Free. (once you buy the macs that is)

I wonder what the actual lifetime amortized cost will be.

Re: macOS 26.2 enables fast AI clusters with RDMA over Thunderbolt

#137

Post by dghlsakjg »

heavyset_go wrote: post #100
dghlsakjg wrote: post #93

Earlier quoted context omitted.

Huh? Home PCs are as cheap as they’ve ever been. Adjusted for inflation the same can be said about “home use” Macs. The list price of an entry level MacBook Air has been pretty much the same for more than a decade. Adjust for inflation, and you get a MacBook air for less than half the real cost of the launch model that is massively better in every way. A blip in high end RAM prices has no bearing on affordable home c…

Home calculators are cheap as they've ever been, but this era of computing is out of reach for the majority of people. The analogous PC for this era requires a large amount of high speed memory and specialized inference hardware.

What regular home workload are you thinking of that the computer I described is incapable of?

You can call a computer a calculator, but that doesn’t make it a calculator.

Can they run SOTA LLMs? No. Can they run smaller, yet still capable LLMs? Yes.

However, I don’t think that the ability to run SOTA LLMs is a reasonable expectation for “a computer in every home” just a few years into that software category even existing.

Re: macOS 26.2 enables fast AI clusters with RDMA over Thunderbolt

#138

Post by [deleted] »

Eggpants wrote: post #103
bigyabai wrote: post #67

Earlier quoted context omitted.

The lack of official Linux/BSD support is enough to make it DOA for any serious large-scale deployment. Until Apple figures out what they're doing on that front, you've got nothing to worry about.

Not sure I understand, Mac OS is BSD based. https://en.wikipedia.org/wiki/Darwin_(operating_system)

[deleted]

Re: macOS 26.2 enables fast AI clusters with RDMA over Thunderbolt

#140

Post by [deleted] »

awnihannun wrote: post #29
simonw wrote: post #3

I follow the MLX team on Twitter and they sometimes post about using MLX on two or more joined together Macs to run models that need more than 512GB of RAM. A couple of examples: Kimi K2 Thinking (1 trillion parameters): https://x.com/awnihannun/status/1986601104130646266 DeepSeek R1 (671B): https://x.com/awnihannun/status/1881915166922863045 - that one came with setup instructions in a Gist: https://gist.github.com/…

For a bit more context, those posts are using pipeline parallelism. For N machines put the first L/N layers on machine 1, next L/N layers on machine 2, etc. With pipeline parallelism you don't get a speedup over one machine - it just buys you the ability to use larger models than you can fit on a single machine. The release in Tahoe 26.2 will enable us to do fast tensor parallelism in MLX. Each layer of the model is…

[deleted]
Post reply on HN