Live data from Hacker News

Add 10 GbE to your system with an M.2 2280 module

cnx-software.com

241–250 of 259 posts

Re: Add 10 GbE to your system with an M.2 2280 module

#241
post #160

Earlier quoted context omitted.

If somebody knows why they should pay a premium for 10GbE, they also know how bad spinning rust is.

Those same people should also know that spinning rust can go very fast as well... My 8 and 12 drive spinning rust pools can read at 500MB/sec easily and get close to GB/sec. But the 10Gbit network is there for the flash pool.

Or say 1Gbps is too low bar. Even single rust drive can fill 1GbE in simple workload.

Re: Add 10 GbE to your system with an M.2 2280 module

#242

Earlier quoted context omitted.

> windows docker images are incredibly bloated They are no-where near that size ...

base image for .NET Framework is 10GB. they only get larger from there. Largest on my machine right now is showing to take 20GB. Granted there are a lot of shared layers between images, so the amount of space on-disk is much less than 20GB. But if you don't have any of the layers already? You're looking at 20GB easy.

And you have to start from scratch each month if you want to keep up with Windows Updates -- they apply to the base image.

Re: Add 10 GbE to your system with an M.2 2280 module

#243

Earlier quoted context omitted.

Lots of consumer motherboards don't have a lot of PCIe slots wider than x1 (which is not enough for 10GigE), but they often have multiple NVMe capable M.2 slots, that this card would work with. In my home server for instance, which is built on a consumer μATX board, the wider PCIe slots are filled with a GPU and HBA, which leaves no way to add 10GigE without spending a lot of money on a new motherboard and CPU - or f…

Port expanders are a thing... Unless you plan to be needing full bandwidth to your GPU at the same time as full network bandwidth, you shouldn't have issues. I never really understood why motherboards didn't spend a few extra dollars and make all ports 16x ports (switched), so that you can use full CPU bandwidth to any one device and not have to mess with different types of port for different devices.

Because the switch (PLX) became too expensive by acquisition. Maybe there are cheaper alternatives for now? but is it support Gen4?

Re: Add 10 GbE to your system with an M.2 2280 module

#244
post #28

Earlier quoted context omitted.

Spinning HDDs are faster than this as well, 250-300 MB/s sequential read seems common per drive, and in a home setup you might have a couple of those in a RAID-1 giving 500-600 MB/s read BW.

> couple of those in a RAID-1 giving 500-600 MB/s If you are talking about only two drives than no. Multiple drives can give you more throughput, but at that point it is RAID-10.

Right, for 2x single stream sequential read throughput + redundancy I guess you need somehing like Linux f2 RAID10 mode. Otherwise reading every other stripe per disk would kill you due to seeks or 50% wasted readahead.

Re: Add 10 GbE to your system with an M.2 2280 module

#245

Earlier quoted context omitted.

I have gigabit internet (downstream only... hoping for fiber one day for symmetric up/down), and I run wired gigabit ethernet for desktops and fixed devices to free up the wifi. For 99% everyday use it works just fine. Externally I hit 80+ MB/s on internet download (aggregate, very few sites saturate my downstream bandwidth), and internally I hit 90+ MB/s to and from my NAS. The only time I wish it was faster is when…

I’ve got google fiber 1 gig up and down. It’s plenty fast for external content but internally I’ve been annoyed at the network perf to my NAS so I got a secondary network card just to go from desktop to Nas at 10GbE speeds and it’s been great and it skips the need to overhaul the entire network stack.

Did that take any complicated setup to make the routing work?

Re: Add 10 GbE to your system with an M.2 2280 module

#246

It's interesting to see just how slow faster ethernet standards have been. Feels like I've had gigabit ethernet at a pretty low cost for nearly 20 years now, but faster than that has been pretty esoteric outside of the datacentre. I guess there really isn't much demand for faster than gigabit speeds even now (outside of servers?)

Maybe once gigabit internet service becomes more widely available, but even for something like a consumer NAS you’re probably bottlenecked by something else no?

I checked performance on my server locally vs accessing the server as a NAS.

The simple spinning hard-drive was already faster than gigabit ethernet, and the NAS access speed was 90-95% of ethernet. So I am prerty sure that NAS is the bottleneck here. And this is on a WD-red 5400RPM harddrive, nothing special.

Re: Add 10 GbE to your system with an M.2 2280 module

#247

Earlier quoted context omitted.

> windows docker images are incredibly bloated They are no-where near that size ...

base image for .NET Framework is 10GB. they only get larger from there. Largest on my machine right now is showing to take 20GB. Granted there are a lot of shared layers between images, so the amount of space on-disk is much less than 20GB. But if you don't have any of the layers already? You're looking at 20GB easy.

This isn't true...

I just downloaded: https://hub.docker.com/_/microsoft-dotnet-framework

It was about 2.7 / 2.8 GB in size.

Total size on disk was 8.7GB but we are talking about download here.

I would say you should be moving to .Net 6 it's been several years now that .Net Core has been out so there is no excuse.

None of this is an excuse to get a 10Gbps internet connection to your house. I have a 500Mbps connection and with the corporate firewall my laptop gets 100-200Mbps connection and pulling those containers was like 10 minutes.

Edit: In case anyone thinks I'm lying:

https://devblogs.microsoft.com/dotnet/we-made-windows-server...

Re: Add 10 GbE to your system with an M.2 2280 module

#248

Earlier quoted context omitted.

If you do not need a cable, there are m.2 to 4x PCIe (open slot, so you can insert 8x and 16x cards) for $5.

I wasn't able to find any of those that support PCI-e 3.0. Both the Marvel and Intel controllers discussed in this thread are PCI-e 3.0.

https://www.aliexpress.com/item/1005002996748461.html

↑ this one does support gen3. Anyhow, with 4x@gen2 you’ll be fine at 10G speeds (I checked). Gen3 is strictly needed only if you want 2 (or 4) ports, or 25G...

Re: Add 10 GbE to your system with an M.2 2280 module

#249
post #83

Earlier quoted context omitted.

10GbE and 100GbE on fiber is quite cheap and easy now - but 99% of consumers and people doing ordinary stuff have no capability or interest in doing fiber. You can terminate cat5e or cat6 with $25 in hand tools... I think what's new is the prevalence now of 2.5 and 5GBaseT ethernet chips that are cheap enough companies are starting to build them into any $125+ ATX motherboard. At short lengths even old crappy cat5e h…

2.5GBASE-T and 5GBASE-T are designed to work across 100 meters of Cat 5e. Nothing crappy about it! :)

That said...

https://www.cablinginstall.com/sponsored/berk-tek/article/16...

... points out that 5GBASE-T needs 200 MHz channel bandwidth which is past what Cat 5e is specified for. So perhaps for runs approaching 100 meters or in noisy environments, Cat 5e won't be reliable for 5GBASE-T after all.

Re: Add 10 GbE to your system with an M.2 2280 module

#250
post #103

Earlier quoted context omitted.

I wanted to upgrade portions of my home network to multigig because Comcast is giving us 1.4Gbps and I wanted to use it. At least for me, 2.5 switches were way to expensive so I ended up with used Intel 10G cards connected with DACs to a cheap 5 port Mikrotik 10G switch. One 10GBase-T RJ45 SFP+ hooks into the modem.

"1.4Gbps" and probably 16Mbps upstream on DOCSIS3.1 channel bonding I'd rather have a 100Mbps home DIA circuit that was symmetric and seriously reliable, than the typical Comcast circuit.

It's generally around 35-40Mbps upstream and between 600 and 1.2Gbps observed downstream, depending on the time of day.

I would also very much rather have a symmetric connection than what I have now. We are within range of a fiber tap for Comcast's 3Gbps symmetric ftth service but construction costs are too high right now for them to do it. I keep asking every six months regardless. There's no other game in town as far as I can tell.

Comcast Business will run a 100Mbps symmetric fiber for ~$750/mo, maybe that would be a thing to do to get fiber in the door and then switch back to residential after the contract earns out.

Post reply on HN