Live data from Hacker News

10Gb/s Ethernet: what I did to get it working in my home

gilesthomas.com

81–90 of 180 posts

Re: 10Gb/s Ethernet: what I did to get it working in my home

#81
post #57
post #29

Earlier quoted context omitted.

Wow, and at essentially the same price!

Yeah, the new ones have gotten much cheaper it seems. About a year ago they were ~2x.

I think a lot of patents have recently expired for 10GbaseT. I’ve noticed it getting a lot more accessible.

Re: 10Gb/s Ethernet: what I did to get it working in my home

#82
Using SFP+ modules for 10GbE is interesting, I deployed a full 10GbE network in my home connected by 5Gbit/s symmetric fiber from AT&T and a 1Gbit/s x 100mbit/s DOCSIS 3.1 connection from Spectrum as backup, I did this around August of last year and used UniFi/Ubiquiti for everything. Their new XG and XGS products all support 10GbE natively, without needing SFP+ modules to connect. I'm using the UDM Pro Max with an XGSPON ONU flashed with the community firmware from PON Wiki (see https://pon.wiki/category/att/ ) and the 2.5GbE port as my second WAN link to the Spectrum modem. I'm not doing any sort of bonding/load balancing, just failover so I'm almost always just using AT&T. There's a short DAC cable connecting the UDM Pro Max to a Switch Pro XG 24-POE which serves 10GbE to the rest of the house. The only SFP+ module involved is the XGSPON ONU. Most of the devices in the house connect to WiFi served by the UniFi U7 Pro XGS APs, which take 10GbE w/ POE and give off WiFi 7.

I pretty much always get at least 4600Mbit/s both directions over AT&T, and generally cap it out. Spectrum typically gives me at least 800Mbit/s down. The ISPs are definitely the bottleneck because neither provides dedicated infrastructure from each house to the CO, instead you have some sort of aggregation point which has a shared backhaul and that means you compete for resources, but having the largest plan on each at least gives you traffic priority.

Realistically, I don't need any of this, I was doing just fine with normal gigabit fiber with a 2.5GbE network before I moved last year, but it's nice knowing everything is as fast as possible in my path and that eliminates congestion and local network resource contentions as causes when a problem arises, I know it's pretty much always upstream of me somewhere.

Re: 10Gb/s Ethernet: what I did to get it working in my home

#83
post #35
post #26

Unfortunately the blog didn't link to the SFP+ module they're using, but everyone should know there's effectively 2 different generations of 10gbit sfp+ to ethernet^H10BASE-T modules. The old gen, labeled as 30 meters, draws ~3 W, and gets extremely hot (to the point it'll usually cause link flaps), and the newer gen, usually labeled as 100m or 80m, draws ~1.5 W, and runs much, much cooler. Example of the new gen: ht…

Thanks! 10Gb Eth is insane for exactly this reason (optical SFP+ modules are way cheaper and more reliable)

What's really insane is 40G - nobody wants it so you can buy kit for next to nothing. I got a 72×QSFP Cisco Nexus (with EVPN & VXLAN) for £50 inc P&P.

Re: 10Gb/s Ethernet: what I did to get it working in my home

#84

I'm extremely happy after upgrading my network to 10gbit copper ethernet. It was much more expensive than I thought it should be, but worth it even if I only max it occasionally. Now I can easily fully saturate my 10gbit ethernet doing a first Time Machine backup or transferring files to my M.2 SSD NAS which saves me waiting rime and is satisfying to watch. It's wild to me that 10gbit isn't the norm by now and tech p…

Heh it's honestly wild to me anyone needs over a gig. My work has a one gig fiber line supporting hundreds of employees and usage generally remains below 10%. The high expense of 10gig is, in part, because it isn't widely necessary and the people buying it are willing to pay extra.

> The high expense of 10gig is, in part, because it isn't widely necessary and the people buying it are willing to pay extra.

I think the price has more to do with where you live and how the market is structured than how necessary it is. In Japan where there is competition between ISPs, I pay about $40/mo for 10Gbps.

Routers have also come down in price to where they are pretty affordable for consumers. I use a Ubiquiti Cloud Gateway Fiber [1] which has three 10Gbs ports (two SPF+, one 10GbE) for $279. A TP-Link router [2] with an upstream 10GbE port and 2.5GbE LAN ports and Wi-Fi 7 is about $140. 2.5 GbE NICs have become cheap and ubiquitous and could commonly be found on $150 mini pcs (before memory and SSD prices went crazy).

Yeah it's more than more than most people need, but I definitely appreciate having the increased speed when downloading 50GB games, uploading 200GB files to YouTube, or backing up files to the cloud. I've probably never maxed out the full 10Gbps, but exceeding 1Gbps is pretty easy in relatively common use cases.

[1] https://store.ui.com/us/en/category/cloud-gateways-compact/c...

[2] https://amzn.asia/d/03EKpC8E

Re: 10Gb/s Ethernet: what I did to get it working in my home

#85
post #35

Earlier quoted context omitted.

Thanks! 10Gb Eth is insane for exactly this reason (optical SFP+ modules are way cheaper and more reliable)

I don't agree that it is insane. It is less efficient than ideal, but 10gbit over copper is not necessarily dangerously hot or difficult to power. I have a MikroTik CRS304-4XG-IN on my office desk with three out of four ports at 10gbit and it is perhaps 20 degrees above ambient on the outside. Warm but not hot. Passively cooled design. A normal Windows laptop runs hotter than that when idle.

I have several 48 port 10G copper switches in my computer room with 40G uplinks. Dell OS9 jobbies. They are a bit noisy, but they don't actually run that hot. I also have some HPE 1G 48 porters and they run roughly the same temperature.

I used to daily drive Gentoo on my laptop. Lovely in the winter!

Re: 10Gb/s Ethernet: what I did to get it working in my home

#86

I'm extremely happy after upgrading my network to 10gbit copper ethernet. It was much more expensive than I thought it should be, but worth it even if I only max it occasionally. Now I can easily fully saturate my 10gbit ethernet doing a first Time Machine backup or transferring files to my M.2 SSD NAS which saves me waiting rime and is satisfying to watch. It's wild to me that 10gbit isn't the norm by now and tech p…

> It was much more expensive than I thought it should be

For 10G with copper, short runs will work fine on most anything (10m), with longer working ok on cat-5e cable (20 - 30m or so).

So, if you have existing wiring, especially in wall, just give it a try first, before gutting things. Most 10g ethernet adapters give you BER stats to see the error rate.

And, if there are issues, multi-gig may be an option, where it will drop down to 5 or 2.5 gbps, which is same as 10G, but with a reduced symbol table to handle the lower SNR.

Re: 10Gb/s Ethernet: what I did to get it working in my home

#87

I'm extremely happy after upgrading my network to 10gbit copper ethernet. It was much more expensive than I thought it should be, but worth it even if I only max it occasionally. Now I can easily fully saturate my 10gbit ethernet doing a first Time Machine backup or transferring files to my M.2 SSD NAS which saves me waiting rime and is satisfying to watch. It's wild to me that 10gbit isn't the norm by now and tech p…

> It's wild to me that 10gbit isn't the norm by now

10G was too big of a step up from 1G. The expense and power required made it unattractive. Only recently have the interfaces for 10G over twisted pair become reasonably low power.

2.5G and even 5G are in a much better spot. It's where I recommend most people start as a default.

Re: 10Gb/s Ethernet: what I did to get it working in my home

#88
post #77
post #35

Earlier quoted context omitted.

Thanks! 10Gb Eth is insane for exactly this reason (optical SFP+ modules are way cheaper and more reliable)

The main issue is that it is ancient . The cost, power, and length issues meant that it wasn't exactly well-received by the datacenter market back in 2006(!) when it was first released: DAC was the far more attractive option for a link from server to top-of-rack, and fiber was obviously superior (if not plain required ) for anything beyond a hop to the next aisle over. This left an incredibly tiny market, so obviousl…

Right, but for example my fiber ONT only has an rj45 port so I'm stuck using one of these for the link to my UDM Pro. The router and my core switch use cheap DACs.

Re: 10Gb/s Ethernet: what I did to get it working in my home

#89
post #85

Earlier quoted context omitted.

I don't agree that it is insane. It is less efficient than ideal, but 10gbit over copper is not necessarily dangerously hot or difficult to power. I have a MikroTik CRS304-4XG-IN on my office desk with three out of four ports at 10gbit and it is perhaps 20 degrees above ambient on the outside. Warm but not hot. Passively cooled design. A normal Windows laptop runs hotter than that when idle.

I have several 48 port 10G copper switches in my computer room with 40G uplinks. Dell OS9 jobbies. They are a bit noisy, but they don't actually run that hot. I also have some HPE 1G 48 porters and they run roughly the same temperature. I used to daily drive Gentoo on my laptop. Lovely in the winter!

Yes the noise is my main complaint for my 10G switch. I didnt expect that high frequency part.

Re: 10Gb/s Ethernet: what I did to get it working in my home

#90

I'm extremely happy after upgrading my network to 10gbit copper ethernet. It was much more expensive than I thought it should be, but worth it even if I only max it occasionally. Now I can easily fully saturate my 10gbit ethernet doing a first Time Machine backup or transferring files to my M.2 SSD NAS which saves me waiting rime and is satisfying to watch. It's wild to me that 10gbit isn't the norm by now and tech p…

> It's wild to me that 10gbit isn't the norm by now 10G was too big of a step up from 1G. The expense and power required made it unattractive. Only recently have the interfaces for 10G over twisted pair become reasonably low power. 2.5G and even 5G are in a much better spot. It's where I recommend most people start as a default.

Yeah, 10GbE-over-copper switches are SO expensive. I bought a Ubiquiti enterprise switch for my home; 10Gb uplink and the copper ports are split between 2.5G and 1G. It's fine because only 1 or 2 clients can even talk 10G, and those are all across the house on Cat5 links anyway so they only negotiate to 2.5 even on a 10G port.

As much as I wanted to "future proof" by having a 24 port 10GbE switch... why? I'll just wait and buy one when I have a use for it.

Post reply on HN