Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

10Gb/s Ethernet: what I did to get it working in my home

gilesthomas.com

Post reply on HN
Print view View original

71–80 of 180 posts

Re: 10Gb/s Ethernet: what I did to get it working in my home

#71

Post by lukasgraf »

TexanFeller wrote: post #68
lukasgraf wrote: post #56

Earlier quoted context omitted.

I can't remember the exact phrasing, but are you talking about the error message that essentially says: "The Tardis is broken. Your backup has diverged into an entirely separate timeline, and I have no way of reconciling it. You may now sacrifice an entire weekend to do an initial backup again."? I've been on a lucky streak for several years now, where I haven't gotten that one on any of my devices. "Preparing backup…

> "Preparing backup..." taking an unreasonable amount of time is a regular occurrence, TM heavily throttles disk I/O used for backing up in order to ensure that normal user activity isn't affected. That makes it appear that TM is dramatically slower than you would expect which greatly annoys me. This becomes obvious after you run this command which will make both the preparing and transferring phases go closer to the…

> TM heavily throttles disk I/O used for backing up

That makes sense, and I usually quite like that behavior. I barely ever notice an impact when backups are running.

However, this is happening every time on one machine (Intel iMac), and semi-regularly on another one (M3 MBP), after a fresh restart, giving mds_stores some time to settle down, and the most recent backup just hours ago, with no significant changes on disk since.

In a situation like that, I would expect the "Preparing backup..." stage to just take a second to create an APFS snapshot, and maybe a minute to diff that snapshot against the remote state. But not 10+ minutes.

But thank you for the hint about that sysctl parameter! I will certainly give this a try.

Re: 10Gb/s Ethernet: what I did to get it working in my home

#72

Post by saltcured »

ocdtrekkie wrote: post #33
TexanFeller wrote: post #16

I'm extremely happy after upgrading my network to 10gbit copper ethernet. It was much more expensive than I thought it should be, but worth it even if I only max it occasionally. Now I can easily fully saturate my 10gbit ethernet doing a first Time Machine backup or transferring files to my M.2 SSD NAS which saves me waiting rime and is satisfying to watch. It's wild to me that 10gbit isn't the norm by now and tech p…

Heh it's honestly wild to me anyone needs over a gig. My work has a one gig fiber line supporting hundreds of employees and usage generally remains below 10%. The high expense of 10gig is, in part, because it isn't widely necessary and the people buying it are willing to pay extra.

Depends a lot on your work type and your prior exposure. If you only work "locally" and upload/download rarely, you may be way less demanding of your network than if you actually do distributed work with remote storage, high-bandwidth communicating tasks, etc.

Over 20 years ago, I was used to having 1g LAN for basic workstations and laptops in an office setting and probably 10-20g uplink from the building (shared by hundreds of staff). I also used 1g at home for my very small LAN between laptop, desktop, and SAN functions. But, my home ISP links were often terrible, such as 128k ADSL or even just a tethered GPRS phone at some points.

You end up with entirely different work styles when you have these different resource constraints.

Re: 10Gb/s Ethernet: what I did to get it working in my home

#73

Post by saltcured »

globular-toast wrote: post #58
saltcured wrote: post #32

Earlier quoted context omitted.

Cat 6 for sure. I thought there was some extra tweak beyond that, but I probably misremembered after all this time. Perhaps it was just that it was plenum-rated..

You might be thinking of cat6a which officially supports 10G over long lengths, unlike cat6.

Nah, it was definitely around 2006 which seems to be before cat 6a was ratified. So I couldn't have bought a reel marked that way...

Re: 10Gb/s Ethernet: what I did to get it working in my home

#74

Post by myrandomcomment »

ocdtrekkie wrote: post #67
TexanFeller wrote: post #61

Earlier quoted context omitted.

1Gb Internet service seems low these days, much less 1Gb LAN. I have 3Gb Google Fiber service and actually get 2+ for individual downloads from some internet services like Steam. Even at 2Gb it's annoying to wait tens of minutes for 100+ GiB games to download. If I go on vacation I come home with 10s of GiBs of photos and videos on multiple devices that start syncing with cloud storage. During the day I need to pull…

This is one of those things where I just have to express that a lot of the HN crowd is entirely divorced from the reality the rest of the world experiences. ;) Nearly nobody has multigig anything in the home, a probably surprisingly large percentage of business networking is 1gig LAN or less. And most people would not notice the difference if they did. I am glad it works for you, but everyone else most certainly does…

I disagree. I pay $120 a month for 5Gbps symmetric connection. I could upgrade that to 10Gbps for 2x, but there is no reason at this point. Even the local max from the cable company is more than 1Gbps, 1.2Gbps down / ~300Mbps up for around $80. Everything is streaming now. I work from home, on video calls. My better half will be watching something the on AppleTV streaming, the kid will be doing the same. I have Backblaze running to do backups to the cloud. 3 different laptops that will run TimeMachine backups to the NAS. The AppleTVs also have the Infuse app on them to stream local video files from the NAS. The security cameras are a constant 60Mpbs 24/7/365 to the NVR. The laptops can push a gig wireless and 2.5Gbps when plugged into the Thunderbolt docks. It is not clear that I need 10Gbps everywhere, but it has its uses. The NAS is at 10G. The link from the main switch to the router is 10G. The 3 APs in the house are at 2.5G and the 2 outside are at 1G. There was a noticeable difference when after I right sized the shared links paths up from 1G. When I say noticeable it both perceived and measured. I used to work doing switch bring up and competitive testing so I have a pretty good idea how this all comes together. Given that a reasonably cheap set of APs can now handle clients at above 1G and internet speeds in some areas being above 1G, moving to at least 2.5G in places is useful and not divorced from reality. I am in tech, but I have help my not tech friends upgrade APs, et.al. for their normal everyday home use cases and they have all been quite happy with the change.

Not being divorced from reality is the only reason I have not dropped $5K on the new Dream Machine Beast that was just released and have not swapped out my Enterprise 48 PoE (1st gen.) for the newest version that has 12 10G-BaseT ports.

Re: 10Gb/s Ethernet: what I did to get it working in my home

#75

Post by jdprgm »

xxpor wrote: post #26

Unfortunately the blog didn't link to the SFP+ module they're using, but everyone should know there's effectively 2 different generations of 10gbit sfp+ to ethernet^H10BASE-T modules. The old gen, labeled as 30 meters, draws ~3 W, and gets extremely hot (to the point it'll usually cause link flaps), and the newer gen, usually labeled as 100m or 80m, draws ~1.5 W, and runs much, much cooler. Example of the new gen: ht…

If you want to buy the cheaper old ones and are concerned about heat just add a usb fan. I have the same mikrotik switch in the post and 2 sfp to rj45 + this fan https://www.amazon.com/dp/B00G059G86?th=1 sitting on top and it makes a dramatic difference in temp.

Re: 10Gb/s Ethernet: what I did to get it working in my home

#77

Post by crote »

thefz wrote: post #35
xxpor wrote: post #26

Unfortunately the blog didn't link to the SFP+ module they're using, but everyone should know there's effectively 2 different generations of 10gbit sfp+ to ethernet^H10BASE-T modules. The old gen, labeled as 30 meters, draws ~3 W, and gets extremely hot (to the point it'll usually cause link flaps), and the newer gen, usually labeled as 100m or 80m, draws ~1.5 W, and runs much, much cooler. Example of the new gen: ht…

Thanks! 10Gb Eth is insane for exactly this reason (optical SFP+ modules are way cheaper and more reliable)

The main issue is that it is ancient.

The cost, power, and length issues meant that it wasn't exactly well-received by the datacenter market back in 2006(!) when it was first released: DAC was the far more attractive option for a link from server to top-of-rack, and fiber was obviously superior (if not plain required) for anything beyond a hop to the next aisle over.

This left an incredibly tiny market, so obviously beyond the initial investment very little effort was put into developing new products for it. So now the prosumer market is hitting the limit of 1Gbps, 2.5GBASE-T and 5GBASE-T (both based on the techniques of 10GBASE-T, by the way) are becoming the norm, and suddenly network vendors remember that box of ancient 10GBASE-T transceiver chips that has been collecting dust in their warehouse.

Aaand suddenly you've got people buying what they think is a brand-new technology, but which is actually designed and manufactured using technology from a decade and a half earlier, and 10GBASE-T gets a bad name for being "hot" and "power-hungry". Turns out it is actually reasonably well-behaved if you actually make use of modern technology!

I expect we'll be using it for quite a while. 25GBASE-T and 40GBASE-T are even deader: A standard from 2016, which a decade later doesn't have a single available product? Mandatory switching to Cat8 cabling - and only a 30 meter range??? And no forwards-looking compatibility? Yeeaah, no thanks.

Re: 10Gb/s Ethernet: what I did to get it working in my home

#78

Post by jdprgm »

ocdtrekkie wrote: post #33
TexanFeller wrote: post #16

I'm extremely happy after upgrading my network to 10gbit copper ethernet. It was much more expensive than I thought it should be, but worth it even if I only max it occasionally. Now I can easily fully saturate my 10gbit ethernet doing a first Time Machine backup or transferring files to my M.2 SSD NAS which saves me waiting rime and is satisfying to watch. It's wild to me that 10gbit isn't the norm by now and tech p…

Heh it's honestly wild to me anyone needs over a gig. My work has a one gig fiber line supporting hundreds of employees and usage generally remains below 10%. The high expense of 10gig is, in part, because it isn't widely necessary and the people buying it are willing to pay extra.

I have it more for the fast nas access and being able to treat nas disks as more or less the same performance as if they were directly sata in my machine. Significantly less so about the external network aspect.

Re: 10Gb/s Ethernet: what I did to get it working in my home

#79

Post by apelapan »

toast0 wrote: post #59
apelapan wrote: post #5

I might have been lucky, but in the one home and one office were I've connected 10gbit switches and PCIe cards, it has just worked. Especially the office was a nice surprise, because it is at least 20 meters (probably more) of unknown cabling and at least one unknown patch panel between the utility closet where the NAS lives and the desk area. The cables were run 15 years ago, so I expected it to be cat 5, but clearl…

> The cables were run 15 years ago, so I expected it to be cat 5, but clearly not. Did you check the jackets? I've got Cat5 (no e) marked cable running 10gbaseT. A lot of cable exceeds the specs on its jacket and specified wire provides enough signal to noise for the provided length in dense conduit. When you have shorter runs, without dense wiring, lesser cable can work.

I can't see the cable jackets. There's just outlets in the walls. I ask building management to make a link between outlet X and Y and then they make it happen via one or more patch panels that I don't know where they are. :-)

People downthread has written, cat6 was already the goto thing 15+ years ago so perhaps that is what they put in when the office space was built. It is a well built office!

Re: 10Gb/s Ethernet: what I did to get it working in my home

#80

Post by TacticalCoder »

thefz wrote: post #37

For me the threshold has always been "can I stream a 4K movie from the NAS downstairs or from my seed box". No real need for anything above. Still I ran 10Gb single mode fiber in all the ducts.

> For me the threshold has always been "can I stream a 4K movie from the NAS downstairs or from my seed box".

So basically, and this is not a snark, you're happy with 100 Mbit/s?

Post reply on HN