Live data from Hacker News

Building a Budget Homelab NAS Server

mtlynch.io

141–150 of 316 posts

Re: Building a Budget Homelab NAS Server

#141

About 7 years ago there was an Amazon sale ($300) on Lenovo TS140 towers with the low-powered Xeon chip and ECC RAM and 4 drive bays. Ever since I've been unable to find a similar price point for the same quality, but wanted a backup server. I recently got a Raspberry Pi 4 (8GB model) and external USB hard drive (8TB) mirrored with a s3backer volume on backblaze B2 for about $300 total, and as a backup server it's fa…

Appreciate the insights on your S3 backup solution.

I will mention that I am one of those folks with a TS140. Love that it's a power sipper. I maxed out the processor and memory, as well as loading it up with two 10 TB rust disks and two 512 GB SSDs.

Re: Building a Budget Homelab NAS Server

#142

This article seems right up my alley, so here are some thoughts: - ZFS is pretty amazing in it's abilities, with it ushering in the age of software RAID over hardware RAID - ZFS shouldn't be limited to FreeBSD. The Linux port has come quite a long way. I'd advise you to use PPA over repo though, as many key features are missing from the version on repos. - TrueNAS is more targeted towards enterprise applications. If…

I know software raid is better overall but are there any advantages to hardware raid anymore? Is it just worse at everything?

Re: Building a Budget Homelab NAS Server

#143

Earlier quoted context omitted.

Yes but there’s a price/reliability/performance trade off. Also, with disks that big failures become qualitatively different. For example, when a disk fails in a mirror, the bigger the disk size the higher the chance the 2nd disk will have unreadable blocks.

This is what ZFS scrubbing is for. If a drive develops unreadable sectors, ZFS will alert you.

If the drives are in a RAID ZFS will not only alert you but fix the corruption from parity on other disks.

Re: Building a Budget Homelab NAS Server

#144
With over 25 years of large scale *nix sysadmin experience: please please please don't fall in to the trap of thinking RAID5/Z is a good idea. It almost never is.

The number 1 trap you fall in to is during rebuild after a failed drive. In order to rebuild every byte on every other drive has to be read. On massive arrays this process invariably throws up additional errors, however this time you might not have the parity data to recover it. This process continues in a snowballing situation. This problem is exacerbated by using unsuitable drives. This author seems to have chosen well, but many choose to select drives for capacity over reliability in a quest for the most TB usable possible. A few years ago there was also the scandal of the WD Red drives that were totally unsuitable for RAID usage.

And to make matters worse there is the performance impact. Writing consists of 4 operations: read, read parity, write, write partity. That gives a /4 penalty on the sum of your arrays drives IOPS.

RAID6/Z2 gives you slight relief from the above risk, however at the increased cost of an additional performance hit (a /6 penalty)

If going RAID(Z), it is generally considered best practice to go for a model that includes a mirror. There are decisions to be made whether you stripe mirrors or mirror a stripe. Personally my preference for reducing complexity and improving quick rebuild is to stripe across mirrors. So that is RAID10. You pair your drives up in mirrors, and then you stripe across those pairs. The capacity penalty is 50%. The performance penalty is close to zero.

The author also chose to skip a write buffer (ZIL) drive. This, imo, is a mistake. They are a trivial cost to add (you only require a capacity that gives you the maximum amount of data you can write to your array in 15 seconds (tunable)) and they offer a tremendous advantage. As well as gaining the benefit of SSD IOPS for your writes you also save wear on your data array by coalescing writes in to a larger chunk and buy yourself some security against power cuts etc as faster IOPS give you a reduced likelihood of coinciding with an environmental issue. And if you are especially worried you can add them as a mirrored pair.

You can also add SSDs as a cache (L2ARC) drive (I think the author missed this in their article) to speed up reads. In the case of the authors use case this would really help with things like media catalogs etc as well as buffering ahead when streaming media. The ARC in ZFS always happens, and the L1 is in RAM, but a L2ARC is very beneficial.

The author did comment on RAM for the ARC and sizing this. ZFS will basically use whatever you give it in this regard. The really heavy use case is if you turn on deduplication but that is an expensive and often unnecessary feature. (An example good use case is a VDI server)

Last tip for ZFS: turn on compression. On a modern CPU it's practically free.

Re: Building a Budget Homelab NAS Server

#145

This article seems right up my alley, so here are some thoughts: - ZFS is pretty amazing in it's abilities, with it ushering in the age of software RAID over hardware RAID - ZFS shouldn't be limited to FreeBSD. The Linux port has come quite a long way. I'd advise you to use PPA over repo though, as many key features are missing from the version on repos. - TrueNAS is more targeted towards enterprise applications. If…

I know software raid is better overall but are there any advantages to hardware raid anymore? Is it just worse at everything?

I would go as far as to say "hardware raid" these days is limiting, expensive, and less performant that can be achieved with software RAID.

Re: Building a Budget Homelab NAS Server

#146

This article seems right up my alley, so here are some thoughts: - ZFS is pretty amazing in it's abilities, with it ushering in the age of software RAID over hardware RAID - ZFS shouldn't be limited to FreeBSD. The Linux port has come quite a long way. I'd advise you to use PPA over repo though, as many key features are missing from the version on repos. - TrueNAS is more targeted towards enterprise applications. If…

> - ZFS shouldn't be limited to FreeBSD. The Linux port has come quite a long way. I'd advise you to use PPA over repo though, as many key features are missing from the version on repos.

Agreed. Also for anyone using NixOS, I've found its ZFS support is first class and easy to set up:

https://www.reddit.com/r/NixOS/comments/ops0n0/big_shoutout_...

Re: Building a Budget Homelab NAS Server

#147

This article seems right up my alley, so here are some thoughts: - ZFS is pretty amazing in it's abilities, with it ushering in the age of software RAID over hardware RAID - ZFS shouldn't be limited to FreeBSD. The Linux port has come quite a long way. I'd advise you to use PPA over repo though, as many key features are missing from the version on repos. - TrueNAS is more targeted towards enterprise applications. If…

> I don't recall why, but I've heard that RAIDZ should be avoided, in favor of stripped mirrors.

Most people care about random IO (also once your filesystem has been populated and in use for a while, true linear IO really ceases to be due to fragmentation.) Striped arrays lose random IO performance as drive count goes up; an array of mirrored pairs gains random IO performance. This is less of an issue with tiered storage and cache devices, especially given you almost have to work to find an SSD less than 256GB these days.

You can only upgrade a zdev by upgrading all its drives; it's a lot nicer cash-flow-wise to gradually upgrade a mirrored pair here and there, or upgrade exactly how many pairs you need to for the space you need.

With RAID-Z you have a drive fail and pray a second doesn't fail during the resilver. With RAID-Z2 you can have any two drives fail. With mirrors you can lose 50% of your drives (provided that they're the right drives.)

Re: Building a Budget Homelab NAS Server

#148

This article seems right up my alley, so here are some thoughts: - ZFS is pretty amazing in it's abilities, with it ushering in the age of software RAID over hardware RAID - ZFS shouldn't be limited to FreeBSD. The Linux port has come quite a long way. I'd advise you to use PPA over repo though, as many key features are missing from the version on repos. - TrueNAS is more targeted towards enterprise applications. If…

>- ZFS shouldn't be limited to FreeBSD. The Linux port has come quite a long way. I'd advise you to use PPA over repo though, as many key features are missing from the version on repos.

FreeBSD migrated from own ZFS to OpenZFS so you have single ZFS implementation in BSD and Linux https://openzfs.github.io/openzfs-docs/Getting%20Started/Fre...

Post reply on HN