Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Building a Budget Homelab NAS Server

mtlynch.io

Post reply on HN
Print view View original

261–270 of 316 posts

Re: Building a Budget Homelab NAS Server

#261

Post by justsomehnguy »

Anthony-G wrote: post #215
simonjgreen wrote: post #145

Earlier quoted context omitted.

I would go as far as to say "hardware raid" these days is limiting, expensive, and less performant that can be achieved with software RAID.

I’m in the process of setting up a home server after buying a pair of matching 3TB Western Digital “Red” disks. I plan on installing them in a HPE ProLiant MicroServer G7 Server / HP Micro G7 N40L that I was gifted a couple of years ago. Even though it comes with a hardware RAID, I was considering setting up RAID 1 using Linux software RAID. However, according to the Linux Raid Wiki ¹, Hardware RAID 1 is better than…

> > This is in fact one of the very few places where Hardware RAID solutions can have an edge over Software solutions - if you use a hardware RAID card, the extra write copies of the data will not have to go over the PCI bus, since it is the RAID controller that will generate the extra copy.

This is not a thing to bother (especially after we moved from PCI to PCI-E) for a home user.

The only great thing about HW RAID is what in case your primary drive fail-but-not-fail-completely, ie it would still be seen in the BIOS and BIOS would try to boot from it (but it wouldn't be able to, because drive is half-dead) is what for the BIOS a controller presents a single device and so it would allow booting from a healthy drive.

But again, if this is not a server in a remote oil digging site served twice a year by air (been there, done that) this is not a thing to bother for a home user.

> the possibility of the hardware controller failing is a worry, so I’d need to have a backup strategy for the backup disks

If you use a basic mirror (striped or not) the recovery process is straightforward - for a simple mirror just stick it in any other system/controller, for a striped you would need GetDataBack or R-Studio or just find a newer model of the RAID card from the same vendor.

In your case I would advise to have a single disk in the ODD bay as a boot/system drive and use both your HDDs as an LVM PVs, without fdisk shenanigans. If you/when you decide to upgrade/replace disks the migration would be just a couple of commands like

    pvcreate /dev/sdc
    vgextend your_vg /dev/sdc
    pvmove /dev/mapper/your_vg__your_lv /dev/sda /dev/sdc
    vgreduce /dev/sda

Re: Building a Budget Homelab NAS Server

#263

Post by Nursie »

KennyBlanken wrote: post #147
AviationAtom wrote: post #119

This article seems right up my alley, so here are some thoughts: - ZFS is pretty amazing in it's abilities, with it ushering in the age of software RAID over hardware RAID - ZFS shouldn't be limited to FreeBSD. The Linux port has come quite a long way. I'd advise you to use PPA over repo though, as many key features are missing from the version on repos. - TrueNAS is more targeted towards enterprise applications. If…

> I don't recall why, but I've heard that RAIDZ should be avoided, in favor of stripped mirrors. Most people care about random IO (also once your filesystem has been populated and in use for a while, true linear IO really ceases to be due to fragmentation.) Striped arrays lose random IO performance as drive count goes up; an array of mirrored pairs gains random IO performance. This is less of an issue with tiered sto…

> You can only upgrade a zdev by upgrading all its drives

This is no longer the case, or at least, should no longer be the case soon. The ability to add drives to a zpool has been announced, and will trickle through to stable before too long.

Re: Building a Budget Homelab NAS Server

#264

Post by viraptor »

InvaderFizz wrote: post #246
viraptor wrote: post #244

Earlier quoted context omitted.

Why do you have bitrot to begin with? That's what scheduled scrubbing is for. You could of course by very unlucky and have a drive fail and the other get corruption on the same day, but check how often you find issues with scrubbing and tell me how likely that scenario is.

I've had hundreds of drives in hundreds of terabytes of appliances over years. URE and resilver is a common occurrence, as in every monthly scrub across 200+ drives. This isn't 200 drives in a single array, this is over 4 appliances geographically distributed. The drives have been champs overall, they're approaching an average runtime of about 8 years. During that 8 years we've lost about 20% of the drives in various…

> monthly scrub

Are they used 24/7 at high iops? Why not nightly scrub?

Re: Building a Budget Homelab NAS Server

#265

Post by fomine3 »

gjulianm wrote: post #167
AviationAtom wrote: post #119

This article seems right up my alley, so here are some thoughts: - ZFS is pretty amazing in it's abilities, with it ushering in the age of software RAID over hardware RAID - ZFS shouldn't be limited to FreeBSD. The Linux port has come quite a long way. I'd advise you to use PPA over repo though, as many key features are missing from the version on repos. - TrueNAS is more targeted towards enterprise applications. If…

> - TrueNAS is more targeted towards enterprise applications. If you want good utility as a home user then give Proxmox or the like a look. Then you can make it into more than just a NAS (if you're open to it). I have questions about this. I'm thinking of building my own NAS server, and I don't know which OS to use. On the one hand it looks like people recommend TrueNAS a lot, which is nice now that they have a Linux…

Proxmox is my way. It supports newer ZFS version and support term for major version is fine. I put NAS functionality on LXC container that passthru'd host's filesystem.

Re: Building a Budget Homelab NAS Server

#266

Post by foobarian »

Anthony-G wrote: post #215
simonjgreen wrote: post #145

Earlier quoted context omitted.

I would go as far as to say "hardware raid" these days is limiting, expensive, and less performant that can be achieved with software RAID.

I’m in the process of setting up a home server after buying a pair of matching 3TB Western Digital “Red” disks. I plan on installing them in a HPE ProLiant MicroServer G7 Server / HP Micro G7 N40L that I was gifted a couple of years ago. Even though it comes with a hardware RAID, I was considering setting up RAID 1 using Linux software RAID. However, according to the Linux Raid Wiki ¹, Hardware RAID 1 is better than…

Funny, I have the identical setup with a small SSD for /, and two of the 3TB Reds for my media/music/files. I have seen enough pain from HW Raid controller setups (custom drivers, configs, long recovery) that HW Raid was right out. Then, for soft raid I had bad experiences in the past where the system refused to come up with one disk in the mirror pair bad. I'm sure there is a way to configure this properly but at this point I said fuck it and just mirror one of the WDs by hand to the other via rsync. Think hard (or even test): what actions will need to happen when one of your disks fails? And second, do you need backup or RAID more? Fat-finger deleting a file on a RAID-backed file system will leave you with no backup unless it has snapshots.

Some notes.

- Not having Raid really doesn't matter. The primary purpose seems to be to save a little bit of space by clever checksumming or increase read performance from parallel operation, but none of this is valuable to me.

- I use ext4. I think it would make sense to move to a snapshot-capable system to make the periodic rsync-backups more correct (I don't even bother dropping to r/o mode, since it's fairly static data).

- What really keeps me up at night: bit flips silently corrupting files. I think btrfs or ZFS are supposed to solve this through constant background checksumming. I really need a periodic process to checksum every file and take action on exceptions. Note that RAID will not help you with this.

- This has worked pretty well so far. Twice already (over 12 years) have I had one disk in the pair fail, upon which time I would order a new pair (they were bigger/cheaper by that point and I figured the other one would fail soon) and rebuild the server.

Re: Building a Budget Homelab NAS Server

#267

Post by fulafel »

KennyBlanken wrote: post #166
InvaderFizz wrote: post #158

Earlier quoted context omitted.

> any read error on any of the remaining disks is a lost/corrupted file. That is the meat of it. With traditional RAID it is the same issue, except you never know it happens because as long as the controller reads something, it's happy to replicate that corruption to the other disks. At least with ZFS, you know exactly what was corrupted and can fix it, with traditional RAID you won't know it happened at all until yo…

You don't need to fix anything. ZFS automatically self-heals an inconsistent array (for example if one mirrored drive does not agree with the other, or if a parity drive disagrees with the data stripe.) ZFS does not suffer data loss if you "suffer a total disk failure." I have no idea where you're getting any of this from.

The parity cannot be used in the degraded scenario that was under discussion.

See eg here where the increasing disk size vs specified unrecoverable read error rate is explored in relation to the question at hand: https://queue.acm.org/detail.cfm?id=1670144 (in the article Adam Leventhal from Sun, the makers of ZFS, talks about the need for triple parity).

Also, the conclusion "ensure your backups are really working" is an important point irrespective of this question, since you'll also risk losing data due to buggy software, human errors, ransomware, etc.

Re: Building a Budget Homelab NAS Server

#269

Post by Anthony-G »

justsomehnguy wrote: post #261
Anthony-G wrote: post #215

Earlier quoted context omitted.

I’m in the process of setting up a home server after buying a pair of matching 3TB Western Digital “Red” disks. I plan on installing them in a HPE ProLiant MicroServer G7 Server / HP Micro G7 N40L that I was gifted a couple of years ago. Even though it comes with a hardware RAID, I was considering setting up RAID 1 using Linux software RAID. However, according to the Linux Raid Wiki ¹, Hardware RAID 1 is better than…

> > This is in fact one of the very few places where Hardware RAID solutions can have an edge over Software solutions - if you use a hardware RAID card, the extra write copies of the data will not have to go over the PCI bus, since it is the RAID controller that will generate the extra copy. This is not a thing to bother (especially after we moved from PCI to PCI-E) for a home user. The only great thing about HW RAID…

Wow! That's all really informative. I had thought that a non-striped, mirrored disk should be usable if the H/W RAID controller failed (given that all the data is stored on the disk) but I didn’t know for sure. I didn’t think it would also be possible to recover striped disks.

Thanks for the great advice on using LVM and the commands for replacing a disk. I only recently cane across a system that used the disk block device itself as a physical volume – rather than partitioning it via fdisk or parted. Other than using LVM for creating snapshots, I haven't really used it for anything interesting.

Re: Building a Budget Homelab NAS Server

#270

Post by fomine3 »

KennyBlanken wrote: post #183
zamalek wrote: post #125

> Power usage A 500W PSU won't necessarily draw more than a 250W PSU, that is merely its maximum sustained load (what the rest of the system asks for) rating. The Bronze 80+ rating is likely part of the problem here, that indicates what the power draw from the wall is compared to what is being provided to your system. Titanium 80+ would net you about 10% reduction in wall power usage. Keep in mind that manufacturers…

Switch-mode PSUs are very inefficient at the low end of their duty cycle. A 250W 80-bronze PSU for a 60W load will be operating at 25% capacity and 82% efficiency or better. A 500W 80-titanium PSU at 60W will be at around 12% and 90% efficiency or better. So, an 8% difference in minimum required efficiency...for a huge increase in cost. It's much better to buy a high "tier" PSU (for reliability and safety), sized so…

For building NAS, there's a myth that you need good enough PSU to spin up all disks at once (unless you have delay mechanism). I wonder is it still matter now.
Post reply on HN