Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Building a Budget Homelab NAS Server

mtlynch.io

Post reply on HN
Print view View original

161–170 of 316 posts

Re: Building a Budget Homelab NAS Server

#161

Post by Joel_Mckay »

zamalek wrote: post #128
Joel_Mckay wrote: post #84

Earlier quoted context omitted.

The cephfs per-machine redundancy mode is usually the preferred configuration. i.e. usually avoids cramming everything into a single point of failure, buying specialty SAS cards, and poking at live raid arrays to do maintenance. Seen too many people's TrueNAS/FreeNAS installs glitch up over the years to trust the zfs community edition as a sane production choice. ZFS certainly has improved, but Oracle is not generall…

> Oracle BTRFS seems to be maturing nicely, hopefully we can start using it for these types of workloads in the next few years.

BTRFS? you have to be joking... zfs despite its history, it has rarely achieved that level of silliness.

Re: Building a Budget Homelab NAS Server

#162

Post by KennyBlanken »

anarcat wrote: post #154
InvaderFizz wrote: post #130

Earlier quoted context omitted.

A few points from someone with years managing raid and ZFS in arrays all the way up to 50 disks: RAID-Z1 is something I never consider without a solid backup to restore from and a plan to execute that process at least once in the lifecycle of an array. If you suffer a total disk failure of one of those disks in the array, you have likely lost some data. The good news is that ZFS will tell you exactly which files you…

> If you suffer a total disk failure of one of those disks in the array, you have likely lost some data. [...] The reason is, with a total loss of a single disk, any read error on any of the remaining disks is a lost/corrupted file. Wait, what? If a RAID-(z)1 ZFS array loses one disk, there's data loss? I've ran so many RAID-1 and RAID-10 arrays with mdadm that I can't even being to count them, and I had many drive f…

You're not missing anything. They're completely wrong.

In RAID-Z, you can lose one drive or have one drive with 'bit rot' (corruption of either the parity or data) and ZFS will still be able to return valid data (and in the case of bit rot, self-heal. ZFS "plays out" both scenarios, checking against the separate file checksum. If trusting one drive over another yields a valid checksum, it overwrites the untrusted drive's data.)

Regular RAID controllers cannot resolve a situation where on-disk data doesn't match parity because there's no way to tell which is correct: the data or parity.

Re: Building a Budget Homelab NAS Server

#163

Post by idatum »

Sylveste wrote: post #96
idatum wrote: post #94

I use 2 ZFS mirrored 4TB drives mounted on a USB C dual bay device (iDsonix brand) for backing up my ZFS pools. I have a simple script that imports the backup pool and sends snapshots from my main ZFS pools to the backup pool. My question: How do you safely store your physical backup drives/devices? I have a fireproof box, but I don't think it was made for safely storing electronics in the event of a fire.

The same solution only powered by POE and buried in a pelican case filled with dessicant sacks in your back garden

:-)

To be clear I meant the drives backing up the NAS, not the actual NAS.

I think backing up online ultimately is the safest choice, and it takes getting comfortable doing that and being okay with paying a fee. This is for data that I can't lose, like family photos, etc.

I started looking into using rclone directly from my FreeBSD NAS device. rclone seems to support many providers.

Re: Building a Budget Homelab NAS Server

#164

Post by AviationAtom »

aborsy wrote: post #132
vetinari wrote: post #124

Earlier quoted context omitted.

> I'd advise you to use PPA over repo though, as many key features are missing from the version on repos. I would advise using ZFS only with distros that come with it (i.e. Ubuntu, Proxmox), especially if you plan to have your / on it. I wasted too much time on CentOS with ZFS, would not do it again.

Yeah, Ubuntu has done a pretty good job with ZFS on root installation. Zero setup, works out of box. Highly recommend ZFS and Ubuntu with ZFS!

I think for me I am more concerned about trying to get the OS bootable again, if something becomes corrupted on the OS level. Even with MD RAID it came be a bit of a struggle to recover, but ZFS on Root seemed much harder to troubleshoot and repair. Perhaps I am mistaken in this belief though?

Re: Building a Budget Homelab NAS Server

#165

Post by barrkel »

AviationAtom wrote: post #119

This article seems right up my alley, so here are some thoughts: - ZFS is pretty amazing in it's abilities, with it ushering in the age of software RAID over hardware RAID - ZFS shouldn't be limited to FreeBSD. The Linux port has come quite a long way. I'd advise you to use PPA over repo though, as many key features are missing from the version on repos. - TrueNAS is more targeted towards enterprise applications. If…

Raidz (or preferably raidz2) is good for archival / media streaming / local backup and has good sequential read/write performance, while striped mirrors - raid10 - are better for random access read and write and are a little bit more redundant (i.e. reliable), but costs more in drives for the same usuable space.

Raidz needs to read all of every drive to rebuild after a drive replacement while a striped mirror only needs to read one. However if you're regularly scrubbing zfs then you read it all regularly anyway.

Raidz effectively has a single spindle for random or concurrent I/O since a whole stripe needs to be read or written at a time. Raidz also had a certain amount of wastage owing to how stripes round out (it depends on how many disks are in the array), but you still get a lot more space than striped mirrors.

For a home user on a budget raidz2 usually makes more sense IMO, unless you need more concurrent & random I/O, in which case you should probably build and benchmark different configurations.

I've been using zfs for over 10 years, starting with Nexenta, a defunct oddity with Solaris kernel and Ubuntu userland. These days I use Zfs on Linux. I've never lost data since I started.

Re: Building a Budget Homelab NAS Server

#166

Post by KennyBlanken »

InvaderFizz wrote: post #158
anarcat wrote: post #154

Earlier quoted context omitted.

> If you suffer a total disk failure of one of those disks in the array, you have likely lost some data. [...] The reason is, with a total loss of a single disk, any read error on any of the remaining disks is a lost/corrupted file. Wait, what? If a RAID-(z)1 ZFS array loses one disk, there's data loss? I've ran so many RAID-1 and RAID-10 arrays with mdadm that I can't even being to count them, and I had many drive f…

> any read error on any of the remaining disks is a lost/corrupted file. That is the meat of it. With traditional RAID it is the same issue, except you never know it happens because as long as the controller reads something, it's happy to replicate that corruption to the other disks. At least with ZFS, you know exactly what was corrupted and can fix it, with traditional RAID you won't know it happened at all until yo…

You don't need to fix anything.

ZFS automatically self-heals an inconsistent array (for example if one mirrored drive does not agree with the other, or if a parity drive disagrees with the data stripe.)

ZFS does not suffer data loss if you "suffer a total disk failure."

I have no idea where you're getting any of this from.

Re: Building a Budget Homelab NAS Server

#167

Post by gjulianm »

AviationAtom wrote: post #119

This article seems right up my alley, so here are some thoughts: - ZFS is pretty amazing in it's abilities, with it ushering in the age of software RAID over hardware RAID - ZFS shouldn't be limited to FreeBSD. The Linux port has come quite a long way. I'd advise you to use PPA over repo though, as many key features are missing from the version on repos. - TrueNAS is more targeted towards enterprise applications. If…

> - TrueNAS is more targeted towards enterprise applications. If you want good utility as a home user then give Proxmox or the like a look. Then you can make it into more than just a NAS (if you're open to it).

I have questions about this. I'm thinking of building my own NAS server, and I don't know which OS to use. On the one hand it looks like people recommend TrueNAS a lot, which is nice now that they have a Linux version, but I'm not really sure what does it offer over a raw Debian apart from web/configuration and some extra tools? I have quite some experience in running Debian systems and managing RAIDs (not with ZFS but doesn't seem too much of a jump) and I worry that TrueNAS, while nice at the beginning, might end up being limiting if I start to tweak too much (I plan on using that NAS for more things than just storage).

Re: Building a Budget Homelab NAS Server

#168

Post by 1500100900 »

KennyBlanken wrote: post #162
anarcat wrote: post #154

Earlier quoted context omitted.

> If you suffer a total disk failure of one of those disks in the array, you have likely lost some data. [...] The reason is, with a total loss of a single disk, any read error on any of the remaining disks is a lost/corrupted file. Wait, what? If a RAID-(z)1 ZFS array loses one disk, there's data loss? I've ran so many RAID-1 and RAID-10 arrays with mdadm that I can't even being to count them, and I had many drive f…

You're not missing anything. They're completely wrong. In RAID-Z, you can lose one drive or have one drive with 'bit rot' (corruption of either the parity or data) and ZFS will still be able to return valid data (and in the case of bit rot, self-heal. ZFS "plays out" both scenarios, checking against the separate file checksum. If trusting one drive over another yields a valid checksum, it overwrites the untrusted dri…

They mean: lose one drive and have another with bit rot.

Re: Building a Budget Homelab NAS Server

#169

Post by InvaderFizz »

KennyBlanken wrote: post #162
anarcat wrote: post #154

Earlier quoted context omitted.

> If you suffer a total disk failure of one of those disks in the array, you have likely lost some data. [...] The reason is, with a total loss of a single disk, any read error on any of the remaining disks is a lost/corrupted file. Wait, what? If a RAID-(z)1 ZFS array loses one disk, there's data loss? I've ran so many RAID-1 and RAID-10 arrays with mdadm that I can't even being to count them, and I had many drive f…

You're not missing anything. They're completely wrong. In RAID-Z, you can lose one drive or have one drive with 'bit rot' (corruption of either the parity or data) and ZFS will still be able to return valid data (and in the case of bit rot, self-heal. ZFS "plays out" both scenarios, checking against the separate file checksum. If trusting one drive over another yields a valid checksum, it overwrites the untrusted dri…

The situation I laid out was a degraded Z1 array with the total loss of a single disk(not recognized at all by the system), plus bitrot on at least one remaining disk during resilver. Pairity is gone, you have checksum to tell you that the read was invalid, but even multiple re-reads don't give valid checksum.

How does Z1 recover the data in this case other than alerting you of which files it cannot repair so that you can overwrite them?

Re: Building a Budget Homelab NAS Server

#170

Post by barbazoo »

This is really cool. I've been tinkering, trying to get away from Dropbox and repurposed an old server to SMB share a disk that I occasionally rsync with another disk via ssh. I feel like it's not sufficient to protect against errors. What's a reliable, easy to maintain NAS solution for that purpose, Synology?
Post reply on HN