Live data from Hacker News

Reliable 25 Gigabit Ethernet via Thunderbolt

kohlschuetter.github.io

101–110 of 133 posts

Re: Reliable 25 Gigabit Ethernet via Thunderbolt

#102

Earlier quoted context omitted.

The linux implementation is quite poor. Among other issues, your answer is linux treats it as a TCP/IP link- so packets don't have any offloading for checksums, etc. it's also incomplete (ex- you have to physically unplug and replug the cord every time one side loses link- even a reboot). This is my firsthand experence trying to get some tablet motherboards to link up and work as a proxmox cluster w/ TB3 as the link…

I have a 3 node proxmox setup on MS-01s using a 25G Thunderbolt ring for Ceph, and indeed it took a lot of hoops to get it working correctly and reliably. I did manage to get it such that nodes can go up and down without needing to unplug anything, and the dynamic routing works if a node disappears. Performance is pretty good, with a more realistic 20ish gbit/sec.

I figured this was Jeff’s CTO Laboratory. I enjoy your channel. Are you kicking yourself for pulling the trigger on MS-01s now that MS-02 Ultras released?

Re: Reliable 25 Gigabit Ethernet via Thunderbolt

#103

Thunderbolt is basically external PCIe, so this is not so surprising. High speed NICs do consume a relatively large amount of power. I have a feeling I've seen that logo on the board before.

It isn't. There is no sense in which "Thunderbolt is basically external PCIe". Thunderbolt provides a means of encapsulating PCIe over its native protocols, which puts PCIe on the same footing as other encapsulated things like DisplayPort and, for TB4 and later, USB.

Re: Reliable 25 Gigabit Ethernet via Thunderbolt

#104
> the biggest downside of the PX adapter is that it gets really hot, like not touchable hot. Sometimes, either the network connection silently disappeared or (sadly) my Mac crashed with a kernel panic in the network driver. Apple has assured me that this was not a security issue. Other than that, the PX seems to do the job.

Made me chuckle.

Re: Reliable 25 Gigabit Ethernet via Thunderbolt

#105

Any idea why ethernet stagnated in terms of speed? There was a time it was so much faster compared to usb. Now even wifi seems to be faster. Sure one can buy nice ethernet cards and cables, but the reality is that if you grab a random laptop/desktop from best buy and a cable, you are looking at best at a 2.5Gb/s speed.

PCI-E lanes for consumers. Gigabit would saturate the PCI bus, but once you're on PCI-E you only need to give it 1 lane, usually off the chipset.

Servers had a reason to spend for the 10G, 25G and 40G cards which used 4 lanes.

There are 10 Gigabit chips that can run off of one PCI-E 4.0 lane now and the 2.5G and 5G speeds are supported(802.3bz).

Re: Reliable 25 Gigabit Ethernet via Thunderbolt

#106

Ha. I got one of the 10G Thunderbolt adapters a several years ago. And eventually started having problems with Zoom calls around noon. With dropped connections and stuttering. Zoom restarts usually fixed the problem. After it happened 3-4 times, I started debugging. It turned out that we usually get at least a bit of sunlight around noon, as it burns away the morning clouds. And my Thunderbolt box was in direct sunli…

As my old grandad said, there’s nothing so permanent as a temporary repair!

Re: Reliable 25 Gigabit Ethernet via Thunderbolt

#107

Any idea why ethernet stagnated in terms of speed? There was a time it was so much faster compared to usb. Now even wifi seems to be faster. Sure one can buy nice ethernet cards and cables, but the reality is that if you grab a random laptop/desktop from best buy and a cable, you are looking at best at a 2.5Gb/s speed.

The new low-power Realtek chipsets will definitely push 10 GbE forward because the chipset won't be much more expensive to integrate and run than the 2.5Gbps packages. It all comes down to performance per Watt, the availability of cheap switching gear, and the actual utility in an office / home environment. For 10 Gbps, cabling can be an issue. Existing "RJ45"-style Cat 6 cables could still work, but maybe not all of…

But why is the gear progressing so very slowly? Why a 25 year gap between reasonable power 1Gbps and 10Gbps?

And while not every cat6 will do 10, it would still be worth a shot, and devices aren't using 5 instead they're using even less.

Not to mention that cat8 will happily do 40Gbps as long as you can get from your switch to your end devices in 30 meters.

Re: Reliable 25 Gigabit Ethernet via Thunderbolt

#108

Any idea why ethernet stagnated in terms of speed? There was a time it was so much faster compared to usb. Now even wifi seems to be faster. Sure one can buy nice ethernet cards and cables, but the reality is that if you grab a random laptop/desktop from best buy and a cable, you are looking at best at a 2.5Gb/s speed.

> Any idea why ethernet stagnated in terms of speed? There was a time it was so much faster compared to usb. Now even wifi seems to be faster. Practically spoken, a lot of the transfer speed advertised by wifi is marketing hogwash barely backed by reality, especially in congested environments. > Sure one can buy nice ethernet cards and cables, but the reality is that if you grab a random laptop/desktop from best buy…

> Practically spoken, a lot of the transfer speed advertised by wifi is marketing hogwash barely backed by reality, especially in congested environments.

Yes but a hogwash of several gigabits sometimes does give you real-world performance of more than a gigabit.

> Intel doesn't provide many lanes, so manufacturers don't want to waste valuable lanes permanently for capabilities most people don't ever need.

It's been a bunch of years that a single lane could do 10Gbps, and a bunch more years that a single lane could do 5Gbps.

Also don't ethernet ports tend to be fed by the chipset? So they don't really take lanes.

Re: Reliable 25 Gigabit Ethernet via Thunderbolt

#110

Please forgive me for my ignorance, but are there currently any ways of being able to write data down at that speed? I see 2026 PCIe 5.0 NVMe advertising theoretical 14gb/s but not sure how feasible even that is.

25 gigabit/s is 3125 megabyte/s, the SSD in my 4 year old laptop can write at nearly 6000 megabyte/s.
Post reply on HN