Live data from Hacker News

Add 10 GbE to your system with an M.2 2280 module

cnx-software.com

81–90 of 259 posts

Re: Add 10 GbE to your system with an M.2 2280 module

#81
post #45

This would be so much nicer if it was SFP+, 10GBASE-T transceivers are too expensive still IMO.

They're 50USD or so. The main problem seems to be that they use ~3-4W of power each, and it's more than power budget of a single SFP/+ slot (2W or so), and they become v.hot (~70°C), what can lead to overheating of switches.

Yeah, the mikrotiks recommend populating only every other slot if you're using RJ45 transceivers to avoid overheating.

Re: Add 10 GbE to your system with an M.2 2280 module

#82
post #72

I have had a lot of trouble with the Marvell AQtion controllers under Linux. They are supposed to work and are plug and play on modern kernels, but I was never able to resolve a bug where the controller stopped working after a few hours. The only resolution I found was rebooting, which made the controller not very useful. The Intel 10GbE controllers are more expensive, but much better in my experience. For anyone loo…

Same experience with a bunch of Aquantia AQC-107 (ASUS XG-C100C). Had to remove them from a Linux server, it just won't work and botch IPv6 traffic (especially routing advertisement notices ?!). Got Intel x550t2 and all the issues miraculously disappeared.

I have this controller, and I was able to mostly workaround the dying issue by having a cron script ping a network device every minute, and when that fails it restarts the link - `ip l set enp70s0 down; sleep 6; ip l set enp70s0 up`.

But that's acceptable only because that machine has a workload which can tolerate not having network access for a few minutes per day or so.

Re: Add 10 GbE to your system with an M.2 2280 module

#83

It's interesting to see just how slow faster ethernet standards have been. Feels like I've had gigabit ethernet at a pretty low cost for nearly 20 years now, but faster than that has been pretty esoteric outside of the datacentre. I guess there really isn't much demand for faster than gigabit speeds even now (outside of servers?)

10GbE and 100GbE on fiber is quite cheap and easy now - but 99% of consumers and people doing ordinary stuff have no capability or interest in doing fiber. You can terminate cat5e or cat6 with $25 in hand tools... I think what's new is the prevalence now of 2.5 and 5GBaseT ethernet chips that are cheap enough companies are starting to build them into any $125+ ATX motherboard. At short lengths even old crappy cat5e h…

2.5GBASE-T and 5GBASE-T are designed to work across 100 meters of Cat 5e. Nothing crappy about it! :)

Re: Add 10 GbE to your system with an M.2 2280 module

#84
post #36

It's interesting to see just how slow faster ethernet standards have been. Feels like I've had gigabit ethernet at a pretty low cost for nearly 20 years now, but faster than that has been pretty esoteric outside of the datacentre. I guess there really isn't much demand for faster than gigabit speeds even now (outside of servers?)

Lots of reasons: 1. Consumer broadband speeds rarely exceed 1Gbps 2. 1Gbps Local network transfers are seldom slowed by the network (as large file work typically involves HDD still) 3. Where local network transfers are impeded by the network speed the transferring itself isn't a frequent enough and blocking thing that people feel they need to fix it... they just go make a cup of tea once or twice a day 4. There are a…

Another factor: data centres moved to fiber. And fiber is less physically robust and not great for desktop connections or plugging in to a laptop.

10GBASE-T exists, but it turns out pushing 10gbit/s over 100m of twisted pair requires chips that are hot and power hungry. Again, not great for desktops or laptops. And because it’s not used in data centres, there are no economies of scale or trickle down effects.

Gigabit Ethernet and wifi being “good enough” combined with 10gig over twisted pair being expensive and power hungry means that the consumer space has been stuck for a long time.

Re: Add 10 GbE to your system with an M.2 2280 module

#85

It's interesting to see just how slow faster ethernet standards have been. Feels like I've had gigabit ethernet at a pretty low cost for nearly 20 years now, but faster than that has been pretty esoteric outside of the datacentre. I guess there really isn't much demand for faster than gigabit speeds even now (outside of servers?)

[deleted]

Re: Add 10 GbE to your system with an M.2 2280 module

#86
post #77

Earlier quoted context omitted.

It's kind of chicken and egg. It's not worth buying a 10G switch when all or nearly all of your devices are gigabit and it's not worth buying 10G cards for any device when you have a gigabit switch. What you need for the transition is for premium brands to start pushing 10G ports as a feature, e.g. Apple needs to add it to Macbooks and the Mini and start using it to bludgeon competitors who don't have it. Then once t…

I'd say it's worth it as soon as you have a home NAS, it lets you treat it almost as a local drive for any computer that also has 10G.

I have a home NAS, but I think I need at least two, maybe three 10G switches in order to get everything hooked up properly. And then I need gear to actually get 10G on my computers. Sounds a bit expensive especially since the NAS is unfortunately limited to 2x1GbE.

Re: Add 10 GbE to your system with an M.2 2280 module

#87
post #72

Earlier quoted context omitted.

Same experience with a bunch of Aquantia AQC-107 (ASUS XG-C100C). Had to remove them from a Linux server, it just won't work and botch IPv6 traffic (especially routing advertisement notices ?!). Got Intel x550t2 and all the issues miraculously disappeared.

I have this controller, and I was able to mostly workaround the dying issue by having a cron script ping a network device every minute, and when that fails it restarts the link - `ip l set enp70s0 down; sleep 6; ip l set enp70s0 up`. But that's acceptable only because that machine has a workload which can tolerate not having network access for a few minutes per day or so.

Damn that’s nasty. I wish there was a way to flag known issues on every product that contains garbage chips like this.

Re: Add 10 GbE to your system with an M.2 2280 module

#89

Some people are saying that they don't feel the need for > 1 Gbps in a home network. As a counter-point, I'm regularly limited by 1 Gbps PHY limits. During the 2 years of the pandemic I've been working from home with gigabit Internet. Why gigabit? Because that's the "speed limit" of the Ethernet PHY in the fibre broadband box, and also my laptop's ethernet port. My laptop can write to disk at multiple GB per second,…

I don’t see a use case for 10gb Ethernet in your home network in your post, unless I’m missing something? It sounds like your ISP is limited to 1gbps and all your use cases seem to be bottlenecked by it.

Do you have home servers you forgot to mention that you’re uploading docker images to, and those could benefit from 10gb Ethernet?

(For me, I use gigabit Ethernet everywhere at home but sometimes need to transfer large disk images from my desktop to my laptop, and using a thunderbolt cable as a network cable helps here, I can get closer to 2 gigabits of transfer speed before disk writes seem to be the bottleneck.)

Re: Add 10 GbE to your system with an M.2 2280 module

#90
post #84
post #36

Earlier quoted context omitted.

Lots of reasons: 1. Consumer broadband speeds rarely exceed 1Gbps 2. 1Gbps Local network transfers are seldom slowed by the network (as large file work typically involves HDD still) 3. Where local network transfers are impeded by the network speed the transferring itself isn't a frequent enough and blocking thing that people feel they need to fix it... they just go make a cup of tea once or twice a day 4. There are a…

Another factor: data centres moved to fiber. And fiber is less physically robust and not great for desktop connections or plugging in to a laptop. 10GBASE-T exists, but it turns out pushing 10gbit/s over 100m of twisted pair requires chips that are hot and power hungry. Again, not great for desktops or laptops. And because it’s not used in data centres, there are no economies of scale or trickle down effects. Gigabit…

> fiber is less physically robust

Maybe this used to be the case long ago, but I don’t think it’s true today. Personally I’m pretty rough with fiber (having slammed cabinet doors on fiber, looped fiber around posts with a low bend radius, left the ends exposed to dust, etc) and had no issues within a data center. Can’t say the same for copper. Even the cheapest 10km optics have more margin when your link is only 50m.

Oh and bend insensitive fiber is dirt cheap and works just fine when it’s tied into knots.

Post reply on HN