1.5 TB of VRAM on Mac Studio – RDMA over Thunderbolt 5

Post by Hikikomori » Fri, Dec 19, 2025, 11:12 AM UTC

angoragoats wrote: post #51

nicky_nickell wrote: post #48

Earlier quoted context omitted.

how significant is the latency hit?

The OP makes reference to this with a link to a GitHub repo that has some benchmarks. TCP over Thunderbolt compared to RDMA over Thunderbolt has roughly 7-10x higher latency, ~300us vs 30-50us. I would expect TCP over 200GbE to have similar latency to TCP over Thunderbolt. Put another way, see the graphs in the OP where he points out that the old way of clustering performs worse the more machines you add? I’d expect…

Post by mft_ » Fri, Dec 19, 2025, 11:31 AM UTC

theshrike79 wrote: post #6

delaminator wrote: post #3

> Working with some of these huge models, I can see how AI has some use, especially if it's under my own local control. But it'll be a long time before I put much trust in what I get out of it—I treat it like I do Wikipedia. Maybe good for a jumping-off point, but don't ever let AI replace your ability to think critically! It is a little sad that they gave someone an uber machine and this was the best he could come u…

Yea, I don't understand why people use LLMs for "facts". You can get them from Wikipedia or a book. Use them for something creative, write a short story on spec, generate images. Or the best option: give it tools and let it actually DO something like "read my message history with my wife, find top 5 gift ideas she might have hinted at and search for options to purchase them" - perfect for a local model, there's no wa…

Post by pjmlp » Fri, Dec 19, 2025, 11:53 AM UTC

SoftTalker wrote: post #81

donavanm wrote: post #68

Earlier quoted context omitted.

Enterprise never ever mattered, and there arent enough digits available to show your “home lab” use case in the revenue numbers. Xserve, the RAID shelves, and the directory services were kinda there as a half hearted attempt for that late 90-00s AV setup. All of that fell on the cutting room floor once personal devices, esp iphone, was realized. By the time I left in ‘10 the total revenue from mac hardware was like 1…

Seems like I remember the main reason Macs survived as a product at all was because you needed one to develop for iOS. That may be an exaggeration but there certainly was a time when Macs were few and far between outside of creative shops. Certainly they were almost unseen in the corporate world, where now they are fairly common at least in laptops.

Post by pjmlp » Fri, Dec 19, 2025, 12:00 PM UTC

jauntywundrkind wrote: post #90

I really hope AMD or Intel can get on the clue train and respond. Intel in particular has half a decade of having extremely amazing Thunderbolt ports on their mobile chips, built in (alas not present on desktop chips, for shame). There's been not bad but not great thunderbolt host-to-host networking, that TCP can go over, but the system to system connectivity had been a total afterthought, not at all tuned for obviou…

Post by angoragoats » Fri, Dec 19, 2025, 12:12 PM UTC

wtallis wrote: post #53

angoragoats wrote: post #49

Earlier quoted context omitted.

As with most 40+GbE ports, the 400Gbit ports can be split into 2x200Gbit ports with the use of special cables. So you can connect a total of 6 machines at 200Gbit.

Ah, good point. Though if splitter cables are an option, then it seems more likely that the 50G ports could be combined into a 200G cable. Marvell's product brief for that switch chip does say it's capable of operating as an 8x 200G or 4x 400G switch, but Mikrotik may need to do something on their end to enable that configuration.