Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Add 10 GbE to your system with an M.2 2280 module

cnx-software.com

Post reply on HN
Print view View original

131–140 of 259 posts

Re: Add 10 GbE to your system with an M.2 2280 module

#131

Post by jagrsw »

loosescrews wrote: post #5

I have had a lot of trouble with the Marvell AQtion controllers under Linux. They are supposed to work and are plug and play on modern kernels, but I was never able to resolve a bug where the controller stopped working after a few hours. The only resolution I found was rebooting, which made the controller not very useful. The Intel 10GbE controllers are more expensive, but much better in my experience. For anyone loo…

If someone wants to buy additional 10G card, and has ~300 USD to spare I suggest Intel E810 series. The cheapest E810 version is, I believe, E810-XXVDA2 which has 2x28Gb SFP ports (so good for the future), and uses PCIE4, which makes it work with 10Gbps bandwidths even in x1 ports (though the card is physically x8 in size, so you need an open x1 port), and sometimes you have just a lonely x1 on your MB if you use the rest for 3 gfx cards for god-knows-what purpose :)

Re: Add 10 GbE to your system with an M.2 2280 module

#132

Post by rocqua »

vladvasiliu wrote: post #22
vardump wrote: post #17

Earlier quoted context omitted.

Getting stuck to 1 Gbps is somewhat crazy, as even the slowest laptop and PC M.2 SSDs can do 10-20 Gbps easy. Fastest ones 50 Gbps+. But I guess most people don't transfer files in their local networks anymore and use their network purely for internet access.

> But I guess most people don't transfer files in their local networks anymore and use their network purely for internet access. I think this is clearly the case. Most new laptops don't even bother with a wired network port. I've got a new "pro" HP laptop the other day, and it only comes with some cheap Wi-Fi card. And it's not an "entry-level" laptop, and it's thick enough for an RJ-45 plug to physically fit. I also…

I am looking at wifi integrated motherboards, not for the wifi, but for the bluetooth support.

Re: Add 10 GbE to your system with an M.2 2280 module

#133

Post by virtuallynathan »

Jnr wrote: post #120
Jnr wrote: post #118

Earlier quoted context omitted.

For a moment I thought about setting up 10GbE at home, but then I realized there is no point yet because my external connection to internet is limited to 1 gigabit. The only use internally would be central storage server. But all the new NVMe SSD's are faster than 10 gigs, so my internal network would be the bottleneck. The next logical step would be upgrading to 100 gig network, but that is too expensive right now.…

Oh, and I would never actually use the copper cables with RJ45 for anything above 1 gigabit. Those ethernet cards heat up a lot, use too much energy. This would be a bad investment. Get a proper patch or optical cables and use SFP+ instead.

I’ve never had this as a problem, and I use 50-100ft of Cat5e + 10GBaseT SFP+’s… it’s well within the thermal limits of the SFP, which probably uses 2-5W.

Re: Add 10 GbE to your system with an M.2 2280 module

#134

Post by virtuallynathan »

thfuran wrote: post #121
bluedino wrote: post #112

Earlier quoted context omitted.

> Note that servers are 100 Gbps as standard now In what world?

In the world of unlimited budgets, where employees all spend their days pulling gigabits per second of egress from the cloud, apparently.

100GbE NICs are <$500, and 100GbE switch ports are $100-150.

Re: Add 10 GbE to your system with an M.2 2280 module

#135

Post by theandrewbailey »

jiggawatts wrote: post #80

Some people are saying that they don't feel the need for > 1 Gbps in a home network. As a counter-point, I'm regularly limited by 1 Gbps PHY limits. During the 2 years of the pandemic I've been working from home with gigabit Internet. Why gigabit? Because that's the "speed limit" of the Ethernet PHY in the fibre broadband box, and also my laptop's ethernet port. My laptop can write to disk at multiple GB per second,…

A few months ago, I bought 2 10GbE NICs off Ebay to connect my main desktop and basement server directly. Since then, I've never seen a transfer between them that was slower than 120 MB/s, usually about 160-200 MB/s. (no RAID or NVMe to NVMe transfers)

Some advice: make sure you know the form factor of your NICs. I accidentally bought FlexibleLOM cards. They look suspiciously like PCIe x8, but won't quite fit. FlexibleLOM to PCIe x8 adapters are cheap though.

Re: Add 10 GbE to your system with an M.2 2280 module

#136

Post by guywhocodes »

majaxg wrote: post #58

The sonnet solo10g works like a charm with osx and linux with just a thunderbolt port, around 9,8Gbps. Use SFP+ port instead of Ethernet, more options for connection and less power draining.

Been eyeing it for a while now gotta admit. The desktop setup I'm looking at would be pretty expensive with both that and a thunderbolt dock. So annoying I can't get a thunderbolt dock with SFP+ directly instead.

Re: Add 10 GbE to your system with an M.2 2280 module

#137

Post by jabl »

DannyBee wrote: post #129
martinald wrote: post #9

It's interesting to see just how slow faster ethernet standards have been. Feels like I've had gigabit ethernet at a pretty low cost for nearly 20 years now, but faster than that has been pretty esoteric outside of the datacentre. I guess there really isn't much demand for faster than gigabit speeds even now (outside of servers?)

The main thing, i think, is that outside of DAC, most faster speeds are used over fiber. The cabling starts to become an issue otherwise. It's also hot and power hungry over copper. Having gotten into the home fiber side of it for various reasons, it's clear on the fiber side there is plenty of innovation/cost lowering/etc.

I mostly agree, but optical transceivers are still expensive. Been waiting for dirt cheap silicon photonics for a couple of decades now. But it seems they are on track to be a thing, though initially for high-speed networking not home/office type stuff. Maybe one day, sigh.

Re: Add 10 GbE to your system with an M.2 2280 module

#138

Post by ckdarby »

jiggawatts wrote: post #80

Some people are saying that they don't feel the need for > 1 Gbps in a home network. As a counter-point, I'm regularly limited by 1 Gbps PHY limits. During the 2 years of the pandemic I've been working from home with gigabit Internet. Why gigabit? Because that's the "speed limit" of the Ethernet PHY in the fibre broadband box, and also my laptop's ethernet port. My laptop can write to disk at multiple GB per second,…

How do you regularly pull 50-200 GB? This sounds like terrible practices or implementation at work.

I've never seen docker images of that size that can't be shrunk with multistage, or docker shrink. I can see how someone could get to that size with just baking in all their models to the image itself but yuck.

Re: Add 10 GbE to your system with an M.2 2280 module

#139

Post by bradfa »

gjulianm wrote: post #57
walrus01 wrote: post #37

Earlier quoted context omitted.

10GbE and 100GbE on fiber is quite cheap and easy now - but 99% of consumers and people doing ordinary stuff have no capability or interest in doing fiber. You can terminate cat5e or cat6 with $25 in hand tools... I think what's new is the prevalence now of 2.5 and 5GBaseT ethernet chips that are cheap enough companies are starting to build them into any $125+ ATX motherboard. At short lengths even old crappy cat5e h…

Even 100GbE is hardly seen on company datacenters. Yes, it's cheaper than before, but still more expensive than 10G, and that's extra cost multiplied by all the devices that need to have the improved hardware to take advantage. Plus, most servers won't saturate a 10G link without tweaks on the setup. For 100G it's even worse, I think it will take a long time to see them on datacenters outside of core links or for com…

I think the common knowledge that most servers can't saturate a 10Gb Ethernet link is no longer true. In my experience even saturating 25Gb links is rather easy to do when using 9000 byte MTU on mid-tier server hardware.

100Gb links do take some thought and work to saturate, but that's improving at a good rate lately so I expect it'll become more common rather soon.

The main downside to 25Gb and 100Gb links still seems to be hardware pricing. At these speeds, PCIe network adapters and switches get rather expensive rather quick and will make you really evaluate if your situation really demands those speeds. 10Gb SFP+ and copper network adapters and switches are quite inexpensive now in 2022.

Re: Add 10 GbE to your system with an M.2 2280 module

#140

Post by PragmaticPulp »

jiggawatts wrote: post #80

Some people are saying that they don't feel the need for > 1 Gbps in a home network. As a counter-point, I'm regularly limited by 1 Gbps PHY limits. During the 2 years of the pandemic I've been working from home with gigabit Internet. Why gigabit? Because that's the "speed limit" of the Ethernet PHY in the fibre broadband box, and also my laptop's ethernet port. My laptop can write to disk at multiple GB per second,…

As a counterpoint: I had 10G networking and went back to 1G last time I reorganized. Haven’t really missed the 10G for now.

I ditched the 10G because the 10G switch was hot and noisy and had a fan, whereas my 1G equipment was silent and low power. This could be overcome by putting the switch in another room, obviously, but I haven’t missed it enough to go through the trouble yet.

> During the 2 years of the pandemic I've been working from home with gigabit Internet.

Doesn’t this make every other point in your post moot? I’m similarly limited by Gigabit internet, so the only time I benefit from 10G is transfers to/from my local NAS. I realized that I almost never lose time sitting and waiting for NAS transfers to finish, so even that was only a rare benefit.

If I was buying new equipment and building out something new, I’d pick 2.5G right now. It’s significantly faster than 1G but doesn’t come with the heat and noise that a lot (though not all) of 10G switches come with.

I’m sure I’ll go back to 10G some day, but without >1G internet and no local transfer use case, I found the extra hassle of the equipment (laptop adapters, expensive/hot/noisy switches) wasn’t really worth it for me yet.

Post reply on HN