Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Getting 25 Gbps Thunderbolt Ethernet on My Mac Studio

jeffgeerling.com

Post reply on HN
Print view View original

31–40 of 121 posts

Re: Getting 25 Gbps Thunderbolt Ethernet on My Mac Studio

#31

Post by cyberax »

pzmarzly wrote: post #2

One of the YouTube comments pointed out the issue is most likely the lack of support for SMB Direct (RDMA) in macOS. Would be nice to repeat the test on a Windows/Linux laptop.

I have a 25GbE setup using Thunderbolt, kinda similar to the author's. Except that my fan is held together by a piece of string, not a 3D-printed enclosure.

I found that SMB on macOS is just bad. If I try to heavily load it (e.g. by doing mass conversions in my Audible library), it can cause stalls and disconnections.

I managed to find a set of parameters (in /etc/nsmbd.conf) that makes it work more-or-less reliably by trial-and-error: dir_cache_off=yes, file_ids_off=yes, notify_off=yes, mc_on=no.

It's still terrible, though. It's so bad that Linux in a Parallels VM works _better_.

Re: Getting 25 Gbps Thunderbolt Ethernet on My Mac Studio

#32

Post by colechristensen »

teaearlgraycold wrote: post #3

It’s crazy that the Ethernet chips consume more power (presumably - because of the heat) than the processor in the computer. How is driving 25Gb/s such a hot task compared to driving multiple 40Gb/s Thunderbolt ports?

Ethernet is isolated through transformers on the physical layer and they're always sinking power. Thunderbolt is just pushing current directly through.

With ethernet it's for lots of reasons but a big one is not creating ground loops between distant equipment. If you didn't isolate the network cable would be sinking potentially amps of current between distant devices and that just isn't sustainable. You don't have this problem attaching USB between two devices.

Re: Getting 25 Gbps Thunderbolt Ethernet on My Mac Studio

#33

Post by jauntywundrkind »

teaearlgraycold wrote: post #3

It’s crazy that the Ethernet chips consume more power (presumably - because of the heat) than the processor in the computer. How is driving 25Gb/s such a hot task compared to driving multiple 40Gb/s Thunderbolt ports?

It's distance. 25GbaseT is good for 30m+. Thunderbolt is good for 1m without active cables. Making the much more rugged signal is a much more intense job.

And your computer already speaks the PCIe and thunderbolt. It has that probably on CPU. Here we have to receive USB, convert it to Ethernet. On much less up to date process nodes than your CPU.

Third, until very recently for even 10Gbit it meant using a usb4pcie bridge chip which itself was hot, to provide a generic pcie connection, which then talked to a conventional network chip, so it was a double hop: USB pcie nic. I don't know if 25Gbit to USB direct convert is available yet but we've seen a wave of vastly cheaper 10Gbit adapters come out that are single chip USBnic, probably not pcie at all, that are much much cooler. 25Gbit is probably still not here yet, maybe?

Re: Getting 25 Gbps Thunderbolt Ethernet on My Mac Studio

#34

Post by colechristensen »

SlavikCA wrote: post #10

What are typical use cases for needing 25Gbps? Author talks about 4k video, but AI tells me that for "4K production mastering (ProRes 422 HQ, 4:2:2, 10-bit)" the rate is under 2Gbps. I'm not saying it's not needed, - just curious.

If you're editing video you're not just streaming in real time one copy. You have many streams that you're going to want to work around quickly.

Re: Getting 25 Gbps Thunderbolt Ethernet on My Mac Studio

#35

Post by kllrnohj »

SlavikCA wrote: post #10

What are typical use cases for needing 25Gbps? Author talks about 4k video, but AI tells me that for "4K production mastering (ProRes 422 HQ, 4:2:2, 10-bit)" the rate is under 2Gbps. I'm not saying it's not needed, - just curious.

> Author talks about 4k video, but AI tells me that for "4K production mastering (ProRes 422 HQ, 4:2:2, 10-bit)" the rate is under 2Gbps.

That'd be the rate for 1 stream at 1x playback. Multiple streams because of overlays/crossfades/etc... along with playback at >1x (such as during rendering in particular) will change that significantly.

Re: Getting 25 Gbps Thunderbolt Ethernet on My Mac Studio

#36

Post by ssl-3 »

wmf wrote: post #5
teaearlgraycold wrote: post #3

It’s crazy that the Ethernet chips consume more power (presumably - because of the heat) than the processor in the computer. How is driving 25Gb/s such a hot task compared to driving multiple 40Gb/s Thunderbolt ports?

A Mac Studio is over 100 W while a 25G NIC is under 25 W (that's the power limit for a normal PCIe slot).

A normal x16 PCIe slot has always been able to supply up to 75w since the beginning.

Re: Getting 25 Gbps Thunderbolt Ethernet on My Mac Studio

#37

Post by selectodude »

teaearlgraycold wrote: post #3

It’s crazy that the Ethernet chips consume more power (presumably - because of the heat) than the processor in the computer. How is driving 25Gb/s such a hot task compared to driving multiple 40Gb/s Thunderbolt ports?

Something like 5 percent of AI datacenter usage is just optics.

Re: Getting 25 Gbps Thunderbolt Ethernet on My Mac Studio

#38

Post by GeekyBear »

As mentioned in the blog post's footnote, the bottleneck may just be on the NAS side:

> this real-world speed limitation could also be related to my use of my lower-power Arm NAS (running on Ampere Altra, with 32 fairly-slow CPU cores). I was testing writing to an array of fast enterprise NVMe SSDs, so they shouldn't be the bottleneck here.

He was only getting 1 Gigabyte of throughput out of the built in 10 Gigabit Ethernet connection and moving to the 25 Gigabit connection only made a marginal impact on throughput, after all.

Re: Getting 25 Gbps Thunderbolt Ethernet on My Mac Studio

#39

Post by NooneAtAll3 »

AnthonBerg wrote: post #7
teaearlgraycold wrote: post #3

It’s crazy that the Ethernet chips consume more power (presumably - because of the heat) than the processor in the computer. How is driving 25Gb/s such a hot task compared to driving multiple 40Gb/s Thunderbolt ports?

As I understand it, it takes a bit of anger to get electromagnetism to wiggle that fast over copper. Imperfect transistors using their full operational envelope to control a sort of chaotic thing? Fields and stuff.

is thunderbolt not copper?

Re: Getting 25 Gbps Thunderbolt Ethernet on My Mac Studio

#40

Post by belthesar »

GeekyBear wrote: post #38

As mentioned in the blog post's footnote, the bottleneck may just be on the NAS side: > this real-world speed limitation could also be related to my use of my lower-power Arm NAS (running on Ampere Altra, with 32 fairly-slow CPU cores). I was testing writing to an array of fast enterprise NVMe SSDs, so they shouldn't be the bottleneck here. He was only getting 1 Gigabyte of throughput out of the built in 10 Gigabit E…

You're an order of magnitude off there. The 10 Gigabit connection was providing ~ 1 Gigabyte per second of throughput.
Post reply on HN