Is there any way to connect DGX Sparks to this via USB4? Right now only 10GbE can be used despite both Spark and MacStudio having vastly faster options.
macOS 26.2 enables fast AI clusters with RDMA over Thunderbolt
41–50 of 304 posts
Re: macOS 26.2 enables fast AI clusters with RDMA over Thunderbolt
#42Hoping Apple has secured plentiful DDR5 to use in their machines so we can buy M5 chips with massive amounts of RAM soon.
Re: macOS 26.2 enables fast AI clusters with RDMA over Thunderbolt
#43Re: macOS 26.2 enables fast AI clusters with RDMA over Thunderbolt
#44As someone not involved in this space at all, is this similar to the old MacOS Xgrid?
Re: macOS 26.2 enables fast AI clusters with RDMA over Thunderbolt
#45Hoping Apple has secured plentiful DDR5 to use in their machines so we can buy M5 chips with massive amounts of RAM soon.
Apple tends to book its fab time / supplier capacity years in advance
Re: macOS 26.2 enables fast AI clusters with RDMA over Thunderbolt
#46Would this also work for gaming?
No
Re: macOS 26.2 enables fast AI clusters with RDMA over Thunderbolt
#47This implies you'd run more than one Mac Studio in a cluster, and I have a few concerns regarding Mac clustering (as someone who's managed a number of tiny clusters, with various hardware): 1. The power button is in an awkward location, meaning rackmounting them (either 10" or 19" rack) is a bit cumbersome (at best) 2. Thunderbolt is great for peripherals, but as a semi-permanent interconnect, I have worries over the…
For #2, OWC puts a screw hole above their dock's thunderbolt ports so that you can attach a stabilizer around the cord https://www.owc.com/solutions/thunderbolt-dock It's a poor imitation of old ports that had screws on the cables, but should help reduce inadvertent port stress. The screw only works with limited devices (ie not the Mac Studio end of the cord) but it can also be adhesive mounted. https://eshop.macsale…
That screw hole is just the regular locking USB-C variant, is it not?
See for example:
Re: macOS 26.2 enables fast AI clusters with RDMA over Thunderbolt
#48dang I wish I could share md tables.
Here’s a text edition: For $50k the inference hardware market forces a trade-off between capacity and throughput:
* Apple M3 Ultra Cluster ($50k): Maximizes capacity (3TB). It is the only option in this price class capable of running 3T+ parameter models (e.g., Kimi k2), albeit at low speeds (~15 t/s).
* NVIDIA RTX 6000 Workstation ($50k): Maximizes throughput (>80 t/s). It is superior for training and inference but is hard-capped at 384GB VRAM, restricting model size to To achieve both high capacity (3TB) and high throughput (>100 t/s) requires a ~$270,000 NVIDIA GH200 cluster and data center infrastructure. The Apple cluster provides 87% of that capacity for 18% of the cost.
Re: macOS 26.2 enables fast AI clusters with RDMA over Thunderbolt
#49This implies you'd run more than one Mac Studio in a cluster, and I have a few concerns regarding Mac clustering (as someone who's managed a number of tiny clusters, with various hardware): 1. The power button is in an awkward location, meaning rackmounting them (either 10" or 19" rack) is a bit cumbersome (at best) 2. Thunderbolt is great for peripherals, but as a semi-permanent interconnect, I have worries over the…
There are open source MDM projects, I'm not familiar but https://github.com/micromdm/nanohub might do the job for OS upgrades.
Re: macOS 26.2 enables fast AI clusters with RDMA over Thunderbolt
#50I follow the MLX team on Twitter and they sometimes post about using MLX on two or more joined together Macs to run models that need more than 512GB of RAM. A couple of examples: Kimi K2 Thinking (1 trillion parameters): https://x.com/awnihannun/status/1986601104130646266 DeepSeek R1 (671B): https://x.com/awnihannun/status/1881915166922863045 - that one came with setup instructions in a Gist: https://gist.github.com/…
For a bit more context, those posts are using pipeline parallelism. For N machines put the first L/N layers on machine 1, next L/N layers on machine 2, etc. With pipeline parallelism you don't get a speedup over one machine - it just buys you the ability to use larger models than you can fit on a single machine. The release in Tahoe 26.2 will enable us to do fast tensor parallelism in MLX. Each layer of the model is…
But that's only for prefilling right? Or is it beneficial for decoding too (I guess you can do KV lookup on shards, not sure how much speed-up that will be though).