Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Reliable 25 Gigabit Ethernet via Thunderbolt

kohlschuetter.github.io

Post reply on HN
Print view View original

81–90 of 133 posts

Re: Reliable 25 Gigabit Ethernet via Thunderbolt

#81

Post by nunez »

throwaway2037 wrote: post #45

This is an outstanding blog post. Initially, the title did little to captivate me, but the blog post was so well written that I got nerd-sniped. Who knew this little adapter was so fascinating! I wonder if the manufacturer is buying the Mellanox cards used from data center tear-downs. The author claims they can be had for only 20 USD online. That seems too good to be true! Small thing: I just checked Amazon.com: http…

$285 is still an AMAZING price for 25GbE ethernet over TB4. I paid $200 for the Sonnet TB4 10GbE adapter.

Re: Reliable 25 Gigabit Ethernet via Thunderbolt

#82

Post by whatever1 »

Any idea why ethernet stagnated in terms of speed? There was a time it was so much faster compared to usb. Now even wifi seems to be faster.

Sure one can buy nice ethernet cards and cables, but the reality is that if you grab a random laptop/desktop from best buy and a cable, you are looking at best at a 2.5Gb/s speed.

Re: Reliable 25 Gigabit Ethernet via Thunderbolt

#83

Post by throw0101c »

drnick1 wrote: post #73

I used to have an SFP28 Mellanox card in my home server, but went back to a simple 2.5G Ethernet port for the LAN side. The Mellanox card ran hot and needed an extra fan near it to dissipate the heat. It was cool but there was no real benefit other than occasionally when transferring some large files. Until motherboards include SFP ports it's probably not worth the effort at all in home setting; external adaptors lik…

> Until motherboards include SFP ports […]

A micro-ATX motherboard with on-board 2xSFP28 (Intel E810):

* https://download-2.msi.com/archive/mnu_exe/server/D3052-data...

* https://www.techradar.com/pro/this-amd-motherboard-has-a-uni...

Re: Reliable 25 Gigabit Ethernet via Thunderbolt

#84

Post by Palomides »

buildbot wrote: post #60
throwaway2037 wrote: post #45

This is an outstanding blog post. Initially, the title did little to captivate me, but the blog post was so well written that I got nerd-sniped. Who knew this little adapter was so fascinating! I wonder if the manufacturer is buying the Mellanox cards used from data center tear-downs. The author claims they can be had for only 20 USD online. That seems too good to be true! Small thing: I just checked Amazon.com: http…

I believe the author is talking about the OCP (2.0) network card itself, that these adapters internally. The OCP nics are quite cheap compared to pcie - here’s 100GBE for 100! https://ebay.us/m/HMQAph

you can get a 100Gb normal pcie card like a MCX416A for less than $100 if you're willing to flash them

Re: Reliable 25 Gigabit Ethernet via Thunderbolt

#85

Post by fulafel »

whatever1 wrote: post #82

Any idea why ethernet stagnated in terms of speed? There was a time it was so much faster compared to usb. Now even wifi seems to be faster. Sure one can buy nice ethernet cards and cables, but the reality is that if you grab a random laptop/desktop from best buy and a cable, you are looking at best at a 2.5Gb/s speed.

Wireless happened, I'd think. People started using wifi and cellular data for everything, so applications had to adapt to this lowest common denominator, and consumer broadband demand for faster-than-wifi speeds isn't there. Plus operators put all their money into cellular infra leaving no money to update broadband infra.

Re: Reliable 25 Gigabit Ethernet via Thunderbolt

#86

Post by mschuster91 »

whatever1 wrote: post #82

Any idea why ethernet stagnated in terms of speed? There was a time it was so much faster compared to usb. Now even wifi seems to be faster. Sure one can buy nice ethernet cards and cables, but the reality is that if you grab a random laptop/desktop from best buy and a cable, you are looking at best at a 2.5Gb/s speed.

> Any idea why ethernet stagnated in terms of speed? There was a time it was so much faster compared to usb. Now even wifi seems to be faster.

Practically spoken, a lot of the transfer speed advertised by wifi is marketing hogwash barely backed by reality, especially in congested environments.

> Sure one can buy nice ethernet cards and cables, but the reality is that if you grab a random laptop/desktop from best buy and a cable, you are looking at best at a 2.5Gb/s speed.

For both laptops and desktops, PCI lanes. Intel doesn't provide many lanes, so manufacturers don't want to waste valuable lanes permanently for capabilities most people don't ever need.

For laptops in particular, power draw. The faster you push copper, the more power you need. And laptops have even less PCIe lanes available to waste.

For desktops, it's a question of market demand. Again - most applications don't need ultra high transfer rate, most household connectivity is DSL and (G)PON so 1 GBit/s is enough to max out the uplink. And those few users that do need higher transfer rates can always install a PCIe card, especially as there is a multitude of different options to provide high bandwidth connectivity.

Re: Reliable 25 Gigabit Ethernet via Thunderbolt

#87

Post by drnick1 »

omgtehlion wrote: post #80
drnick1 wrote: post #73

I used to have an SFP28 Mellanox card in my home server, but went back to a simple 2.5G Ethernet port for the LAN side. The Mellanox card ran hot and needed an extra fan near it to dissipate the heat. It was cool but there was no real benefit other than occasionally when transferring some large files. Until motherboards include SFP ports it's probably not worth the effort at all in home setting; external adaptors lik…

Yep, these cards need a fan (or any kind of directed air flow). Where did you get "several ms of latency" figure from? I have not measured external card, but may be I should do it... Because cards themselves have latency in range of microseconds, not millis.

I haven't tested this particular Thunderbolt SFP adapter, but my experience with a TP-Link 1Gbps USB adapter is that it adds about 4ms of latency. Far from being unusable and similar to WiFi perhaps, but worse than PCIe cards that should be <1ms.

Re: Reliable 25 Gigabit Ethernet via Thunderbolt

#88

Post by madduci »

kohlschuetter wrote: post #42
madduci wrote: post #40

Earlier quoted context omitted.

Yes, rescan, re-enroll too. But it still shows as disconnected. I don't know if the firmware is completely incompatible, but it is weird that under windows works and in Linux doesn't

Disconnected as in "network"? What PCIe card do you use? Can you update the firmware (maybe from Windows)? Also check the BIOS settings (try setting TB security to "No Security" or "User Authorization") Some OEM Mellanox cards can be cross-flashed to NVIDIA's stock firmware, maybe that's also relevant.

It's embedded in the laptop (a HP ZBook from work). Disconnected as in network. Laptop charges, but signal doesn't work. With Thunderbolt 3 devices, it works. (The card itself is T4).

Re: Reliable 25 Gigabit Ethernet via Thunderbolt

#89

Post by kohlschuetter »

whatever1 wrote: post #82

Any idea why ethernet stagnated in terms of speed? There was a time it was so much faster compared to usb. Now even wifi seems to be faster. Sure one can buy nice ethernet cards and cables, but the reality is that if you grab a random laptop/desktop from best buy and a cable, you are looking at best at a 2.5Gb/s speed.

The new low-power Realtek chipsets will definitely push 10 GbE forward because the chipset won't be much more expensive to integrate and run than the 2.5Gbps packages.

It all comes down to performance per Watt, the availability of cheap switching gear, and the actual utility in an office / home environment.

For 10 Gbps, cabling can be an issue. Existing "RJ45"-style Cat 6 cables could still work, but maybe not all of them.

Higher speeds will most likely demand a switch to fiber (for anything longer than a few meters) or Twinax DAC (for inter-device connects). Since Wifi already provides higher speeds, one may be inclined to upgrade just for that (because at some point, Wireless becomes Wired, too).

That change comes with the complexity of running new cabling, fiber splicing, worrying about different connectors (SFP+, SFP28, SFP56, QSFP28, ...), incompatible transceiver certifications, vendor lock-in, etc. Not a problem in the datacenter, but try to explain this to a layman.

Lastly, without a faster pipe to the Internet, what can you do other than NAS and AI? The computers will still get faster chips but most folks won't be able to make use of the bandwidth because they're still stuck on 1Gbps Internet or less.

But that will change. Swiss Init7 has shown that 25GBps Internet at home is not only feasible but also affordable, and China seems to be adding lots of 10G, and fiber in general.

Fun times ahead.

Re: Reliable 25 Gigabit Ethernet via Thunderbolt

#90

Post by matt-p »

whatever1 wrote: post #82

Any idea why ethernet stagnated in terms of speed? There was a time it was so much faster compared to usb. Now even wifi seems to be faster. Sure one can buy nice ethernet cards and cables, but the reality is that if you grab a random laptop/desktop from best buy and a cable, you are looking at best at a 2.5Gb/s speed.

We have 400Gbe which is certainly faster than USB.. but;

On consumer devices, I think part of the issue is that we’re still wedded to four-pair twisted copper as the physical medium. That worked well for Gigabit Ethernet, but once you push to 5 or 10 Gb/s it becomes inherently expensive. Twisted pair is simply a poor medium at those data rates, so you end up needing a large amount of complex silicon to compensate for attenuation, crosstalk, and noise.

That's doable but the double whammy is that most people use the network for 'internet' and 1G is simply more than enough, 10G therefore becomes quite niche so there's no enormous volume to overcome the inherent issues at low cost.

Post reply on HN