Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

10Gb/s Ethernet: what I did to get it working in my home

gilesthomas.com

Post reply on HN
Print view View original

51–60 of 180 posts

Re: 10Gb/s Ethernet: what I did to get it working in my home

#51

Post by cyberax »

bot403 wrote: post #22
cyberax wrote: post #19

The Mikrotik switch is awesome, and it's still the most compact 10G switch available. You can fix the thermal issue either by adding a small fan (Noctua is great) or by adding more radiators: https://pics.ealex.net/share/UxeSf_AWHLIuc-qzK5zl7JIgQvQDAZh... I've been running it like this in a closed comm box for the last 3 years without any issues. SFP+ modules actually do not use that much power, it's just that it's c…

Cripes. When possible just do fiber and DACs. Faster and much cooler than 10Gbit. 10Gbit uses an absurd amount of power per port thus the need for all that cooling.

DACs are terrible, they are unwieldy and are always either too short or too long.

Fiber is much better, but in my case the house already had Cat5 Ethernet wiring (originally used just for phones!) everywhere.

Another use case for 10G Ethernet is PoE for the WiFi access points. Although you can't use SFP+ modules for that, of course.

Re: 10Gb/s Ethernet: what I did to get it working in my home

#52

Post by ocdtrekkie »

bombcar wrote: post #49
ocdtrekkie wrote: post #46

Earlier quoted context omitted.

That puts you in an extreme minority, even amongst enterprise businesses. Many medium sized enterprises have storage that looks like "a couple dozen TB total" for hundreds of staff. Having 100 TB of storage in your home basement is an even more extreme minority than that. ;) A gigabit connection is more than enough for a 500-person call center today .

Much works on 10/100 if you wanna know the truth about it - but it is really nice to hit full speeds when copying terabytes around.

Agreed. We had IP phones with 100 Mbps switches in between most of our computers and the rest of the network for a long time and very few people noticed. It'd only really be when I was installing a system upgrade or something, and I'd be like "man, it'd be nice if this didn't take an extra two minutes". For normal web access, 100 Mbps and 1000 Gbps aren't really discernable, until you're downloading large files. A lot of 4K streaming videos though, you'll start to feel it quite a bit faster.

And then hilariously, once you go above a gig, the reality is most sites won't serve them to you any faster than that anyways.

Re: 10Gb/s Ethernet: what I did to get it working in my home

#53

Post by TexanFeller »

floathub wrote: post #36
TexanFeller wrote: post #16

I'm extremely happy after upgrading my network to 10gbit copper ethernet. It was much more expensive than I thought it should be, but worth it even if I only max it occasionally. Now I can easily fully saturate my 10gbit ethernet doing a first Time Machine backup or transferring files to my M.2 SSD NAS which saves me waiting rime and is satisfying to watch. It's wild to me that 10gbit isn't the norm by now and tech p…

It's wild to me Time Machine works on your network. Are you just doing "first backups" over and over again, or have you somehow achieved the very rare state where Time Machine can run for, say, a week at a time without falling over? Sorry, this is snarky and off topic, but I'm nostalgic for the days when Time Machine "just worked".

For a very long time I thought Time Machine had become flaky, and I'm sure it's partially to blame, but with my current setup I've literally never observed it corrupt a backup and have to start over.

Before I was using one of the common Synology consumer NAS boxes that are often recommended. The NAS didn't report any errors with the drives or its own hardware, but at least once a month TM would glitch on at least one of my home laptops.

My new setup is an Asus FLASHSTOR 12 Pro Gen2 FS6812X. For a year now it's been running without a single apparent TM glitch while backing up multiple personal laptops and my work laptop. Sometimes I'm plugged in and sometimes I'm backing up over WiFi, but it's always worked.

I tried various recommended settings for the Synology and nothing helped so I strongly suspect that the Synology network protocol(SMB, AFP, etc.) implementations were either buggy themselves or at least not compatible with quirks in Apple's implementations. Synology->Asus fixed all my TM problems instantly and seemingly permanantly!

Re: 10Gb/s Ethernet: what I did to get it working in my home

#54

Post by bombcar »

ocdtrekkie wrote: post #52
bombcar wrote: post #49

Earlier quoted context omitted.

Much works on 10/100 if you wanna know the truth about it - but it is really nice to hit full speeds when copying terabytes around.

Agreed. We had IP phones with 100 Mbps switches in between most of our computers and the rest of the network for a long time and very few people noticed. It'd only really be when I was installing a system upgrade or something, and I'd be like "man, it'd be nice if this didn't take an extra two minutes". For normal web access, 100 Mbps and 1000 Gbps aren't really discernable, until you're downloading large files. A lo…

I found the nicest thing about fiber is I can hit over a gb/s uploading, which is often much more critical-path for whatever I’m doing than a download.

Re: 10Gb/s Ethernet: what I did to get it working in my home

#55

Post by globular-toast »

I put cat6a in my last house and plan to in my current house "just in case". The cable isn't that much more expensive and nobody wants to do cabling again so why not.

But I'll keep using a gigabit switch because I have absolutely no idea what I'd use 10G for. It's crazy that gigabit was affordable for me as a student in the early 00s and between then and now we've gone from DVDs to 4K and it's still plenty fast enough. In fact, most people are happy with WiFi (not me, though).

Re: 10Gb/s Ethernet: what I did to get it working in my home

#56

Post by lukasgraf »

floathub wrote: post #36
TexanFeller wrote: post #16

I'm extremely happy after upgrading my network to 10gbit copper ethernet. It was much more expensive than I thought it should be, but worth it even if I only max it occasionally. Now I can easily fully saturate my 10gbit ethernet doing a first Time Machine backup or transferring files to my M.2 SSD NAS which saves me waiting rime and is satisfying to watch. It's wild to me that 10gbit isn't the norm by now and tech p…

It's wild to me Time Machine works on your network. Are you just doing "first backups" over and over again, or have you somehow achieved the very rare state where Time Machine can run for, say, a week at a time without falling over? Sorry, this is snarky and off topic, but I'm nostalgic for the days when Time Machine "just worked".

I can't remember the exact phrasing, but are you talking about the error message that essentially says: "The Tardis is broken. Your backup has diverged into an entirely separate timeline, and I have no way of reconciling it. You may now sacrifice an entire weekend to do an initial backup again."?

I've been on a lucky streak for several years now, where I haven't gotten that one on any of my devices.

"Preparing backup..." taking an unreasonable amount of time is a regular occurrence, and some edge cases around adjusting TM backup size quotas aren't handled well. But other than that, TM has been working reasonably well for me to back up 10 TB over SMB to a Synology NAS.

My gripe is much more with Apple's abysmal support for SMB and NFS, especially after deprecating AFP. I've been back and forth between them over the years and over several OS versions, and their implementations for both are just terrible.

But over time SMB, for me, proved slightly more stable and performant, with the right tweaks in smb.conf, and authentication and permissions/ownership are easier to deal with than NFS, so I stuck with that.

I also yearn for the days where TM just worked, because somehow, the alternatives are even worse:

- Arq Backup does some things quite well, which is why I use it as part of my 3-2-1. But some of its bugs and implementation decisions just scream "hobby grade" to me.

- Kopia looks interesting, but it's not mature enough yet. Failed for me with absolutely cryptic error messages during repo init both times I tried it, with versions several months apart.

- Restic, Borg / Vorta: Not turnkey enough for me.

Re: 10Gb/s Ethernet: what I did to get it working in my home

#57

Post by xxpor »

CSSer wrote: post #29
xxpor wrote: post #26

Unfortunately the blog didn't link to the SFP+ module they're using, but everyone should know there's effectively 2 different generations of 10gbit sfp+ to ethernet^H10BASE-T modules. The old gen, labeled as 30 meters, draws ~3 W, and gets extremely hot (to the point it'll usually cause link flaps), and the newer gen, usually labeled as 100m or 80m, draws ~1.5 W, and runs much, much cooler. Example of the new gen: ht…

Wow, and at essentially the same price!

Yeah, the new ones have gotten much cheaper it seems. About a year ago they were ~2x.

Re: 10Gb/s Ethernet: what I did to get it working in my home

#58

Post by globular-toast »

saltcured wrote: post #32
loeg wrote: post #13

Earlier quoted context omitted.

Do you mean Cat 6 or Cat 5e? Cat 6e isn't a thing and 6a didn't exist in 2006. 6 was certainly more future-proof at the time, although arguably 5e is still fine even today. (Super-gigabit consumer equipment didn't really exist until the last five years and it's till notably more expensive and less common than gigabit, which runs on 5e just fine.)

Cat 6 for sure. I thought there was some extra tweak beyond that, but I probably misremembered after all this time. Perhaps it was just that it was plenum-rated..

You might be thinking of cat6a which officially supports 10G over long lengths, unlike cat6.

Re: 10Gb/s Ethernet: what I did to get it working in my home

#59

Post by toast0 »

apelapan wrote: post #5

I might have been lucky, but in the one home and one office were I've connected 10gbit switches and PCIe cards, it has just worked. Especially the office was a nice surprise, because it is at least 20 meters (probably more) of unknown cabling and at least one unknown patch panel between the utility closet where the NAS lives and the desk area. The cables were run 15 years ago, so I expected it to be cat 5, but clearl…

> The cables were run 15 years ago, so I expected it to be cat 5, but clearly not.

Did you check the jackets? I've got Cat5 (no e) marked cable running 10gbaseT. A lot of cable exceeds the specs on its jacket and specified wire provides enough signal to noise for the provided length in dense conduit. When you have shorter runs, without dense wiring, lesser cable can work.

Re: 10Gb/s Ethernet: what I did to get it working in my home

#60

Post by chromadon »

godzillabrennus wrote: post #42
ocdtrekkie wrote: post #33

Earlier quoted context omitted.

Heh it's honestly wild to me anyone needs over a gig. My work has a one gig fiber line supporting hundreds of employees and usage generally remains below 10%. The high expense of 10gig is, in part, because it isn't widely necessary and the people buying it are willing to pay extra.

I put 5Gbit internet into my home (fiber) to build my startup. I'm processing terabytes of data. I have over 100TB of storage in my basement. I can regularly saturate my internet connection. That said, I remember well when a 1Gbit connection provided enough bandwidth for a 500-person call center for daily workloads (back about 10 years ago).

Off topic (sorry). Interested to know what your startup is?
Post reply on HN