Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

ZFS: Use mirror vdevs, not RAIDZ

jrs-s.net

Post reply on HN
Print view View original

61–70 of 133 posts

Re: ZFS: Use mirror vdevs, not RAIDZ

#61

Post by Naac »

aidenn0 wrote: post #17
jryb wrote: post #10

My gripe with this is that the author assumes everyone has the same workload and prefers the same set of tradeoffs. I use raidz1 on my home desktop precisely because I would absolutely prefer to have to wait for a resilver than lose data. “So backup your data!” - of course, but that’s just an implicitly larger pool.

TFA suggests using mirrors instead of RAIDZ. There is no case in which RAIDZ1 is more durable against data loss than a mirror is.

Doesn't that ignore the fact that RAIDZ1 gives you more storage space than a mirror?

Re: ZFS: Use mirror vdevs, not RAIDZ

#62

Post by White_Wolf »

tjoff wrote: post #22

I don't agree. And if one of your disks failed, and age was a factor… you’re going to be sweating bullets wondering if another will fail before your resilver completes. So every every single time you loose any drive in a mirror setup you risk all the data on all drives in the entire pool. I sure do hope you aren't on vacation and/or have to order a drive online. It all depends on your use case. For me, raidz3 wins ea…

tbh for my 10 drive home setup I use a Z3 setup. Had dead drives but never more than 1 at a time. I do check them weekly and if they show signs of issues I replace them on the spot though. I do rsync to a second unit in a diff location but (so far) never had to do a restore.

Re: ZFS: Use mirror vdevs, not RAIDZ

#65

Post by sgarland »

As long as you aren't relying on it as backup, it doesn't matter for most use cases.

I'm about to build a zpool consisting of nothing but 3-wide raidz1 vdevs. I can tolerate one drive dying. In the ~8 years or so I've been running a NAS, I've had precisely one drive failure. I am fully aware that survivorship bias is a thing, and anecdotes aren't data, but it's good enough for me.

Anything important is backed up locally and to the cloud. Everything else is merely annoying to have to download again.

Re: ZFS: Use mirror vdevs, not RAIDZ

#66

Post by benlivengood »

gh02t wrote: post #53
minimaul wrote: post #13

Earlier quoted context omitted.

The one thing in the article I agree with without reservation though is that you should always have a backup of your pool! ZFS makes doing good backups easy, with zfs send | zfs receive.

I mean, is this controversial? The problem for a budget conscious home user is: where do you back up to? Doing a complete backup to a remote storage provider isn't a practical option for many users, which leaves the affordable options being to only back up critical data and/or use something like a big USB external drive. Neither of these options are great, so in reality many people end up relying on RAID or mirroring…

I just started playing around with s3blkdevd. S3-compatible storage at backblaze B2 is $0.005/GB-month and I'll see if I can manage a vdev made out of a huge nbd disk. Topical to the article, backblaze has piles of RAID6 and appears to be happy with it.

Another thought has been to find a ZFS-friend in another part of the world and exchange snapshots. Inline encryption makes that a little more viable now.

Right now I'm sending incremental snapshots to AWS Glacier and have another local box mirroring the data (mostly to fully verify the snapshots before sending them to Glacier).

Re: ZFS: Use mirror vdevs, not RAIDZ

#67

Post by benlivengood »

mrlonglong wrote: post #63

I recently built a ZFS raidz2 array with 4 disks. Is the article saying it's better to mirror over four vdevs (or 4 disks?)

I did the same thing as you. I've had to sit through an RMA period on replacing a drive in a home RAID5 and just never wanted anything like that again; every replacement of a disk in a mirrored vdev would feel the same. Data is backed up but it's a headache to have data unavailable.

Re: ZFS: Use mirror vdevs, not RAIDZ

#68

Post by rsync »

aduitsis wrote: post #50
rsync wrote: post #37

We (rsync.net) have several PB of raidz3 deployed all over the world. We use conservatively sized (12-15 drive) vdevs and typically join 3 or 4 of those together to make a pool. I can see getting nervous about raidz2 (sort of analogous to "raid6") after a drive failure ... but losing 4 drives out of 12 in a single raidz3 failure cascade is extremely improbable . We all sleep quite well with this arrangement and have…

UFS2? So you are using FreeBSD?! Yet another reason to support you folks, seriously.

All rsync.net infrastructure is, and has been, built solely on FreeBSD.

Since 2001 ... :)

Re: ZFS: Use mirror vdevs, not RAIDZ

#69

Post by myrond »

I disagree.

With co-located boxes and drop shipped drive replacements the time between a FAULT and the resilver event can be multiple days. Even though the resilver will go faster with a mirror having one disk remaining on a mirror vdev compared to raidz2 (or higher) mirrors will increase risk of data loss irrespective of resilver times because of drop ship drive replacement time.

3TB resilver on my last mechanical drive failure took 6 hours 30 minutes. Plus an additional 3 days for the drive to arrive.

With mirror vdev setups you lose significantly more space as well. If you argue speed is worth it, then I would instead invest that money you saved going with a raidz2 with NVME cache and SLOG.

Users won't notice the resilver event at all with a significant amount of memory and NVME cache + nvme SLOG tuned with a high /sys/module/zfs/parameters/zfs_dirty_data_max and larger than default /sys/module/zfs/parameters/zfs_txg_timeout.

Re: ZFS: Use mirror vdevs, not RAIDZ

#70

Post by aDfbrtVt »

rsync wrote: post #37

We (rsync.net) have several PB of raidz3 deployed all over the world. We use conservatively sized (12-15 drive) vdevs and typically join 3 or 4 of those together to make a pool. I can see getting nervous about raidz2 (sort of analogous to "raid6") after a drive failure ... but losing 4 drives out of 12 in a single raidz3 failure cascade is extremely improbable . We all sleep quite well with this arrangement and have…

I love the backup solution you guys provide with Borg, the pricing is amazing and the product has been rock solid. Any chance of getting similar "expert level" pricing for accounts using ZFS send | receive ?
Post reply on HN