Live data from Hacker News

Getting 25 Gbps Thunderbolt Ethernet on My Mac Studio

jeffgeerling.com

61–70 of 121 posts

Re: Getting 25 Gbps Thunderbolt Ethernet on My Mac Studio

#61

Can't you just put a real PCIe NIC in an eGPU enclosure with a fan? I bet you can solve this problem for like $150 and two craigslist transactions.

Absolutely not. It's another layer of throughput/compatability bingo on top of the already often quite annoying SFP bingo.

If you are unfamiliar, the reality of the situation is quite bad. Slight hardware revisions of adapters/switches, which if you are lucky can be differentiated by manufacture date or serial number sequence, will not recognize known good modules, and if they do you then have to actually test if they can establish link, then let them burn for 24+ hours to see if they can maintain it without dropping (often, because of forward error correction, you will not start to see flapping until enough correctable errors have accumulated and the buffer overflows). Then there are firmware updates where this process must then be repeated.

You would think with how expensive all of this shit is you would simply be able to buy the officially listed as compatible components but my experience with them has been that a shotgun approach is best for your sanity and deadlines, buying sample kits of each module and trying them all, and then once you find the ones that play nice, buying as many of them as is reasonable and testing all of them.

If I had a dollar every time two modules from the same brand, that came in the same box, from the same production run, did not work with one another, I would be able to test your theory for free. Network Engineering is Fun

Re: Getting 25 Gbps Thunderbolt Ethernet on My Mac Studio

#62
post #31
post #2

One of the YouTube comments pointed out the issue is most likely the lack of support for SMB Direct (RDMA) in macOS. Would be nice to repeat the test on a Windows/Linux laptop.

I have a 25GbE setup using Thunderbolt, kinda similar to the author's. Except that my fan is held together by a piece of string, not a 3D-printed enclosure. I found that SMB on macOS is just bad. If I try to heavily load it (e.g. by doing mass conversions in my Audible library), it can cause stalls and disconnections. I managed to find a set of parameters (in /etc/nsmbd.conf) that makes it work more-or-less reliably…

It's always been like this, too! SMB or otherwise. Any company/person that has been using macs long enough has been burned by silently corrupted volumes or etc and has a thorough system of multiple on and offsite backups.

It's not just software companies or home enthusiasts either. There are a lot of ie medical practices or accounting/benefits agencies out there who have sworn off Apple for life after having to spend extreme amounts of money on data recovery to hopefully get their years of records back, usually in a do or die situation like an IRS audit which causes them to discover that their basement was flooding and all of the paper records are destroyed.

Re: Getting 25 Gbps Thunderbolt Ethernet on My Mac Studio

#63

How does Jeff Geerling afford so many Mac Studios?

My default assumption (with no knowledge about this particular case) is that all gear in YouTube videos is sponsored. Too easy to get jealous of all the fancy setups. But I don't want to become a YouTuber so that settles that.

Re: Getting 25 Gbps Thunderbolt Ethernet on My Mac Studio

#64
post #53
post #36

Earlier quoted context omitted.

A normal x16 PCIe slot has always been able to supply up to 75w since the beginning.

A normal non-GPU PCIe x4/x8 slot that you'd use for a NIC provides 25 W.

For a 25G NIC, I’d be perfectly unsurprised if it demanded an x16 slot. I’d be more surprised if it didn’t have a power connector like a video card, though.

Re: Getting 25 Gbps Thunderbolt Ethernet on My Mac Studio

#65

It’s crazy that the Ethernet chips consume more power (presumably - because of the heat) than the processor in the computer. How is driving 25Gb/s such a hot task compared to driving multiple 40Gb/s Thunderbolt ports?

There are no shortcuts in networking. It's quite easy to spend a 2-3x+ premium on a switch or adapter capable of non blocking line rate on every port simultaneously only to then have it fall over to a fraction of that in the form of lots of tiny packets. This is why we have things like QoS, flow control, and ECMP

Re: Getting 25 Gbps Thunderbolt Ethernet on My Mac Studio

#66

Can't you just put a real PCIe NIC in an eGPU enclosure with a fan? I bet you can solve this problem for like $150 and two craigslist transactions.

Thats what I do.

ConnectX-4 and newer are natively supported. I use a ConnectX-4 LX in a $50 Razer CoreX off of craigslist. Works fine, hit similar limits to what Jeff did.

Using an OCP is the neat trick here. MUCH smaller setup.

As far as the other posters comment about Tx's... that is an entirely different topic. I've never had an issue with FS.com optics personally.

Re: Getting 25 Gbps Thunderbolt Ethernet on My Mac Studio

#67
post #28
post #2

One of the YouTube comments pointed out the issue is most likely the lack of support for SMB Direct (RDMA) in macOS. Would be nice to repeat the test on a Windows/Linux laptop.

This is a ConnextX4 on a thunderbolt bridge. This is well trodden territory. Unless some hero of open source has come to bat recently, MacOS doesn't support RDMA with this card or over Ethernet at all for that matter. It's only supported over Thunderbolt. His iperf3 results show that it's able to go full beans on a single 25g port. A cross platform SMB file copy is kind of a meaningless benchmark in this context. IMO…

Commented on another top comment but yeah, I've been running a similar setup for about $200 all in. Using the PCIe LX version takes care of much of the heat issue, and an eGPU dock (Razer Core in my case) takes care of the fan. Craigslist and eBay were the sources.

Speeds aren't any better, but at least it was cheap and fiber.

Re: Getting 25 Gbps Thunderbolt Ethernet on My Mac Studio

#68
For us Plugable 2.5G USB-C to ethernet adapter (bought November 2025), SABRENT USB C to 2.5Gbps ethernet adapter (bought October 2024), UGREEN USB C 2.5G ethernet adapter (also October 2024). In my experience they immediately fall on their face in a speedtest and drop to a couple hundred Mbps, experience packet loss, etc. This is testing across a few modern Apple Silicon Macs.

I now have a WavLink WL-NWU341G 5Gbps dongle with the RTL-8157 that seems to work well, but I've only run a couple of tests-- I am away from my home setup and don't have wired ethernet run to all the rooms where I'm living.

If you do some Googling, there's some good information out there about it, but when I bought the bad ones, I couldn't find any information corroborating my experience.

Re: Getting 25 Gbps Thunderbolt Ethernet on My Mac Studio

#69

How does Jeff Geerling afford so many Mac Studios?

My default assumption (with no knowledge about this particular case) is that all gear in YouTube videos is sponsored. Too easy to get jealous of all the fancy setups. But I don't want to become a YouTuber so that settles that.

The Mac Studio in most of my videos is my own workstation, purchased a while back.

The 4 Mac Studio cluster I tested with Exo and Thunderbolt RDMA was a loaner cluster from Apple. There's no universe in which I could justify buying that.

I always mention any equipment that's provided / loaned / etc. and the nature of my relationships with vendors—I try to also keep this document updated for transparency's sake: https://github.com/geerlingguy/youtube/blob/master/Sponsors....

Re: Getting 25 Gbps Thunderbolt Ethernet on My Mac Studio

#70
post #68

For us Plugable 2.5G USB-C to ethernet adapter (bought November 2025), SABRENT USB C to 2.5Gbps ethernet adapter (bought October 2024), UGREEN USB C 2.5G ethernet adapter (also October 2024). In my experience they immediately fall on their face in a speedtest and drop to a couple hundred Mbps, experience packet loss, etc. This is testing across a few modern Apple Silicon Macs. I now have a WavLink WL-NWU341G 5Gbps do…

I've had a lot of luck with the Plugable 2.5G USB-C adapters, although on some ports they're incredibly unreliable -- I'm guessing maybe something about the power supply?

On a CalDigit TS3 dock, I've had one working off the rear Thunderbolt socket for over a year, but it fails in minutes if plugged into the USB-C one. I've never had issues with a CalDigit Elements (TB4) hub. I've had success plugging them directly into various Chromebooks. On one Windows laptop, they seemed to work on one USB-C port but not the other.

So... reliable once working, but getting there can be a pain.

Post reply on HN