Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Building a Budget Homelab NAS Server

mtlynch.io

Post reply on HN
Print view View original

211–220 of 316 posts

Re: Building a Budget Homelab NAS Server

#211

Post by semi-extrinsic »

walrus01 wrote: post #201

I would recommend anyone building a home NAS like this in 2022 to look into buying some slightly older 10GbE network interfaces on ebay (an intel X520-DA2 with 2 x 10Gbps SFP+ ports can be found for $55) as a PCI-E card. It's not hard to exceed the transfer ability of an ordinary 1000BaseT port to a home switch these days. And if you have just a few powerful workstation desktop PCs it's also worth it to connect them…

Or go a little further and spend ~$200 on a used passive FDR Infiniband switch, $50 per used dual port FDR IB NIC, $40 for a 10 meter fibre optic cable including transceivers at each end.

Then run IP over IB on each host and you have a 56 Gbit network that all your applications will just see as another network interface on each host.

Re: Building a Budget Homelab NAS Server

#213

Post by walrus01 »

semi-extrinsic wrote: post #211
walrus01 wrote: post #201

I would recommend anyone building a home NAS like this in 2022 to look into buying some slightly older 10GbE network interfaces on ebay (an intel X520-DA2 with 2 x 10Gbps SFP+ ports can be found for $55) as a PCI-E card. It's not hard to exceed the transfer ability of an ordinary 1000BaseT port to a home switch these days. And if you have just a few powerful workstation desktop PCs it's also worth it to connect them…

Or go a little further and spend ~$200 on a used passive FDR Infiniband switch, $50 per used dual port FDR IB NIC, $40 for a 10 meter fibre optic cable including transceivers at each end. Then run IP over IB on each host and you have a 56 Gbit network that all your applications will just see as another network interface on each host.

for home use I'd highly recommend sticking with just 10GbE because you're not locking yourself into a dead-end solution of used weird previous gen infiniband stuff.

if you get a $200 switch with a few 10GbE interfaces in it you can easily expand things in the future by trunking vlans to another newer 10GbE capable switch, or connecting to a switch that has multi-gig copper ports for access ports to 2.5/5GBaseT capable desktop PCs and laptops, etc.

$40 for a 10 meter fiber optic cable is a high price when you can buy LC-LC UPC 9/125 duplex 2 meter cables for $3.50 to $4.70 a piece (or a few cents more for additional meters) and connect them between $20 transceivers. no matter what route someone goes with would recommend buying $30-40 of basic fiber connector cleaning supplies.

https://www.fs.com/products/40192.html?attribute=193&id=3026...

if one wants to buy used weird previous gen dead-end stuff there are also tons of very cheap 40GbE mellanox ethernet adapters on ebay with the QSFP to go with them, and if you have a place to put a switch that doesn't matter if it's noisy like in a wiring closet somewhere, cheap 1U switches with 40GbE ethernet ports on them that can also be used as individual 10GbE when broken out.

Re: Building a Budget Homelab NAS Server

#214

Post by magicalhippo »

KennyBlanken wrote: post #147
AviationAtom wrote: post #119

This article seems right up my alley, so here are some thoughts: - ZFS is pretty amazing in it's abilities, with it ushering in the age of software RAID over hardware RAID - ZFS shouldn't be limited to FreeBSD. The Linux port has come quite a long way. I'd advise you to use PPA over repo though, as many key features are missing from the version on repos. - TrueNAS is more targeted towards enterprise applications. If…

> I don't recall why, but I've heard that RAIDZ should be avoided, in favor of stripped mirrors. Most people care about random IO (also once your filesystem has been populated and in use for a while, true linear IO really ceases to be due to fragmentation.) Striped arrays lose random IO performance as drive count goes up; an array of mirrored pairs gains random IO performance. This is less of an issue with tiered sto…

> also once your filesystem has been populated and in use for a while, true linear IO really ceases to be due to fragmentation

Enough concurrent clients doing sequential IO also looks like random IO to a storage server.

Re: Building a Budget Homelab NAS Server

#215

Post by Anthony-G »

simonjgreen wrote: post #145
mardifoufs wrote: post #142

Earlier quoted context omitted.

I know software raid is better overall but are there any advantages to hardware raid anymore? Is it just worse at everything?

I would go as far as to say "hardware raid" these days is limiting, expensive, and less performant that can be achieved with software RAID.

I’m in the process of setting up a home server after buying a pair of matching 3TB Western Digital “Red” disks. I plan on installing them in a HPE ProLiant MicroServer G7 Server / HP Micro G7 N40L that I was gifted a couple of years ago. Even though it comes with a hardware RAID, I was considering setting up RAID 1 using Linux software RAID. However, according to the Linux Raid Wiki¹, Hardware RAID 1 is better than Software RAID 1.

> This is in fact one of the very few places where Hardware RAID solutions can have an edge over Software solutions - if you use a hardware RAID card, the extra write copies of the data will not have to go over the PCI bus, since it is the RAID controller that will generate the extra copy.

I was intending to use these disks for local backup and for storing rips of my extensive CD and DVD collection. As sibling comments mention, the possibility of the hardware controller failing is a worry, so I’d need to have a backup strategy for the backup disks. Since it’s going to be a home server, down-time wouldn’t be a problem.

I don’t have much experience with either hardware or software RAID so I’d welcome any advice.

¹ https://raid.wiki.kernel.org/index.php/Overview#What_is_RAID...

Re: Building a Budget Homelab NAS Server

#216

Post by AdrianB1 »

GekkePrutser wrote: post #189
AviationAtom wrote: post #160

Earlier quoted context omitted.

I think the worst point is vendor lock-in. If your controller fails, and a replacement isn't available, then you may be dead in the water. That kind of goes against the very point of RAID.

But this is why businesses spend so much on their RAID controllers. To make sure they're in warranty and that kind of thing doesn't happen. Incidentally its also pretty great because no business buys them second hand without warranty. So they're usually available for half nothing. I don't use raid cards right now but I do use fibre channel which is also dirt cheap second hand

3 weeks ago I had a controller failure in a manufacturing plant in Latin America. The contract with the manufacturer was to provide a replacement on site in 4 hours. Guess what, 8 hours later the technician with the replacement controller was still on the way.

With TrueNAS I can move my drives to any other computer with the right interface and they will just work. I did this in the past 10 years of using TrueNAS.

Re: Building a Budget Homelab NAS Server

#217

Post by AdrianB1 »

semi-extrinsic wrote: post #211
walrus01 wrote: post #201

I would recommend anyone building a home NAS like this in 2022 to look into buying some slightly older 10GbE network interfaces on ebay (an intel X520-DA2 with 2 x 10Gbps SFP+ ports can be found for $55) as a PCI-E card. It's not hard to exceed the transfer ability of an ordinary 1000BaseT port to a home switch these days. And if you have just a few powerful workstation desktop PCs it's also worth it to connect them…

Or go a little further and spend ~$200 on a used passive FDR Infiniband switch, $50 per used dual port FDR IB NIC, $40 for a 10 meter fibre optic cable including transceivers at each end. Then run IP over IB on each host and you have a 56 Gbit network that all your applications will just see as another network interface on each host.

I bought brand new 10 Gbps NICs ($47/pcs) and switch ($120) for less than that and replacements are readily available. The 20m AOC cables were indeed $40.

A NAS home build of this size will never exceed 10 Gbps, I barely get ~ 2 Gbps out of the spinning disks.

Re: Building a Budget Homelab NAS Server

#218

Post by walrus01 »

AdrianB1 wrote: post #217
semi-extrinsic wrote: post #211

Earlier quoted context omitted.

Or go a little further and spend ~$200 on a used passive FDR Infiniband switch, $50 per used dual port FDR IB NIC, $40 for a 10 meter fibre optic cable including transceivers at each end. Then run IP over IB on each host and you have a 56 Gbit network that all your applications will just see as another network interface on each host.

I bought brand new 10 Gbps NICs ($47/pcs) and switch ($120) for less than that and replacements are readily available. The 20m AOC cables were indeed $40. A NAS home build of this size will never exceed 10 Gbps, I barely get ~ 2 Gbps out of the spinning disks.

the main application where one will see real 10Gbps speeds is if the NAS also has a fast NVME SSD used for smaller file size needs at high speeds...

for instance I have a setup which is meant for working with uncompressed raw yuv420 or yuv422p 1080p and 4K video, there's a 512GB NVME SSD and a 1TB SSD set up as individual JBOD and exposed to the network for video editing scratch file storage, and it will definitely saturate 10GbE.

this is actually needlessly complicated, if/when I build a more powerful desktop pc again I'm just going to put the same work file space storage on a 2TB NVME SSD directly stuck into the motherboard.

Re: Building a Budget Homelab NAS Server

#219

Post by octopoc »

mrb wrote: post #210

I am sure the author will appreciate ditching the proprietary Synology to go instead with a custom ZFS server, as the reliability, recoverability, and feature set of ZFS is quite frankly hard to beat. I have been using ZFS to build my custom NASs for the last... checks notes 17 years. I started back when ZFS was only available on Solaris/OpenSolaris. My builds usually have between 5 and 7 drives (raidz2). However I d…

> For example 14TB drives sell for $240

Where? Also, is it worthwhile to buy hard drives explicitly for NAS when you're using ZFS? For example, Seagate has the IronWolf product line explicitly for NAS and cost more.

Re: Building a Budget Homelab NAS Server

#220

Post by bsder »

Unfortunately, he punted on ECC.

Is there a pointer to someone who does this but actually goes through the ECC grief?

It's really hard to chop through all the ECC "marketing" aka lies from the different motherboard manufacturers.

What's a cost effective CPU/mobo/ECC for NAS?

Post reply on HN