Live data from Hacker News

Reliable 25 Gigabit Ethernet via Thunderbolt

kohlschuetter.github.io

31–40 of 133 posts

Re: Reliable 25 Gigabit Ethernet via Thunderbolt

#32
post #28

I'm surprised you are only getting 20gbit/s. I did not expect PCIe to be be the limiting factor here. I've got a 100gbit cx4 card currently in a PCIe3 X4 slot (for reasons, don't judge) and it easily maxes that out. I would have expected the 25g cx4 cards to be at least able to get everything out of it. RDMA is required to achieve that in a useful way though. Edit: forgot is isn't "true" PCIe but tunneled.

The limitation is Thunderbolt (32 Gbps theoretical limit for PCIe 3 tunneling).

Re: Reliable 25 Gigabit Ethernet via Thunderbolt

#35
post #25

Earlier quoted context omitted.

Yeah, it's because the network card adapter's heatsink is sandwiched between two PCBs. Not great, not terrible, works for me. The placement is mostly determined by the design of the OCP 2.0 connector. OCP 3.0 has a connector at the short edge of the card, which allows exposing/extending the heat sink directly to the outer case. If somebody has the talent, designing a Thunderbolt 5 adapter for OCP 3.0 cards could be a…

I'd be more worried about cooling the transceivers properly.

My optical transceiver gets to around 52 °C (measured via IR camera), well below its design limit, so that's not bad.

If truly concerned, one could use SFP28 to SFP28 cage adapters to have the heat outside the case, and slap on some extra heatsinks there.

Re: Reliable 25 Gigabit Ethernet via Thunderbolt

#36

> reduces temperatures by at least 15 Kelvin, bringing the ambient enclosure temperature below 40 °C, I had to do a double-take when it mentioned Kelvin since That is physically impossible.

reduces temperatures by at least 15 Kelvin == the same as reduces temperatures by at least 15 Celcius.

It 'reduces it by' ... not reduces it TO

Re: Reliable 25 Gigabit Ethernet via Thunderbolt

#37

> reduces temperatures by at least 15 Kelvin, bringing the ambient enclosure temperature below 40 °C, I had to do a double-take when it mentioned Kelvin since That is physically impossible.

Isn't "reduces temperatures by 15 Kelvin" the same as "reduces temperatures by 15 Celsius"?

Re: Reliable 25 Gigabit Ethernet via Thunderbolt

#39
post #38

I still have issues under Linux (Kernel 6.14) and Thinderboldt 4 docking stations. The simply don't get recognised. But this is a cool solution

Thanks! Have you tried the boltctl/rescan setup I mentioned in the post? It should get you going, as long as your Thunderbolt/USB4 setup is correct.

If you're using an adapter card to add Thunderbolt functionality, then your mainboard needs to support that, and the card must be connected to a PCIe bus that's wired to the Intel PCH, not to the CPU.

Re: Reliable 25 Gigabit Ethernet via Thunderbolt

#40
post #38

I still have issues under Linux (Kernel 6.14) and Thinderboldt 4 docking stations. The simply don't get recognised. But this is a cool solution

Thanks! Have you tried the boltctl/rescan setup I mentioned in the post? It should get you going, as long as your Thunderbolt/USB4 setup is correct. If you're using an adapter card to add Thunderbolt functionality, then your mainboard needs to support that, and the card must be connected to a PCIe bus that's wired to the Intel PCH, not to the CPU.

Yes, rescan, re-enroll too. But it still shows as disconnected. I don't know if the firmware is completely incompatible, but it is weird that under windows works and in Linux doesn't
Post reply on HN