ZFS: Use mirror vdevs, not RAIDZ • Hacker News Board

Re: ZFS: Use mirror vdevs, not RAIDZ

#11

Post by Cyph0n » Wed, Dec 09, 2020, 2:51 PM UTC

geerlingguy wrote: post #9

Every time I make this argument (whether regular RAID or ZFS), everyone pulls out their pitchforks and tells me about how they have run RAID 5 or 6 for years and never had a problem, plus "I can have two drives fail! You would lose your array if the wrong two drives fail!" But drive failure is much more likely during an operation like a resilver/resync, and I wonder if the majority of people espousing more risky RAID…

Critical data for the average user typically does not require much space, so the cost of storage overhead in a mirrored ZFS pool is not too bad.

RAIDZ is much more cost efficient when you need to reliably store large content that isn’t really critical. As you’ve noted, media is one good example of this.

In my case, I’m using a RAIDZ1 vdev of 3 disks, with the understanding that things could go south during a resilver.

Re: ZFS: Use mirror vdevs, not RAIDZ

#12

Post by nix23 » Wed, Dec 09, 2020, 2:52 PM UTC

geerlingguy wrote: post #9

Every time I make this argument (whether regular RAID or ZFS), everyone pulls out their pitchforks and tells me about how they have run RAID 5 or 6 for years and never had a problem, plus "I can have two drives fail! You would lose your array if the wrong two drives fail!" But drive failure is much more likely during an operation like a resilver/resync, and I wonder if the majority of people espousing more risky RAID…

I then tell a real horror-story about my past, not so long ago there was a EVA (A SAN from HP) with a freaking out hardware Raid out of nowhere (uptime about 200 days no Firmware update before nothing) it just started to freak out, the story goes on but the lesson...No more HW-Raid/proprietary stuff for me EVER again.

Re: ZFS: Use mirror vdevs, not RAIDZ

#13

Post by minimaul » Wed, Dec 09, 2020, 2:52 PM UTC

minimaul wrote: post #8

As always with RAID-style setups, there’s an inevitable trade off of cost vs capacity vs performance. There’s still a place for RAIDZ/RAIDZ2, and in my opinion that place is storing bulk data that isn’t too heavily accessed or that needs to be stored with an eye towards keeping £/GB down. Yes, mirrors are faster. Yes, mirrors are easier to expand. But across 12 4TB disks that is 24TB instead of 40TB with RAIDZ2 - and…

The one thing in the article I agree with without reservation though is that you should always have a backup of your pool!

ZFS makes doing good backups easy, with zfs send | zfs receive.

Re: ZFS: Use mirror vdevs, not RAIDZ

#14

Post by joekrill » Wed, Dec 09, 2020, 2:54 PM UTC

I've been using ZFS for a few years now and it's just been amazing. Storage is so cheap these days that it's just much simpler and straight-forward to use mirrored vdevs. If a single drive fails, the pool is still completely usable, and all I have to do is swap out the bad drive (when I get around to it, no hurry usually) and resilver. Resilvering can take a while, but everything is still completely usable while it's happening so it just runs in the background and I don't even notice.

I just upgraded a pool of 4 drives from 4TB each to 10TB - I'd never done anything like that before and was rather nervous, but it was just so simple with the mirrored set up - just swap out each drive and resilver one- by-one.

Re: ZFS: Use mirror vdevs, not RAIDZ

#15

Post by nix23 » Wed, Dec 09, 2020, 2:56 PM UTC

Cyph0n wrote: post #11

geerlingguy wrote: post #9

Every time I make this argument (whether regular RAID or ZFS), everyone pulls out their pitchforks and tells me about how they have run RAID 5 or 6 for years and never had a problem, plus "I can have two drives fail! You would lose your array if the wrong two drives fail!" But drive failure is much more likely during an operation like a resilver/resync, and I wonder if the majority of people espousing more risky RAID…

Critical data for the average user typically does not require much space, so the cost of storage overhead in a mirrored ZFS pool is not too bad. RAIDZ is much more cost efficient when you need to reliably store large content that isn’t really critical. As you’ve noted, media is one good example of this. In my case, I’m using a RAIDZ1 vdev of 3 disks, with the understanding that things could go south during a resilver…

Yeah true, in my Home-server i have one mirror (2 disks for OS/Swap etc) and one RaidZ1 (5 Disks for data), for me it's just important that i have no bit-rot etc. Daily backups (restic) are made so no problem whatsoever.

Re: ZFS: Use mirror vdevs, not RAIDZ

#16

Post by blueflame7 » Wed, Dec 09, 2020, 2:59 PM UTC

Mirror pools are dangerous if you have a failing power-supply. All it takes is to lose the wrong two drives together (which are often in the same rack) and you lose a portion of your dataset. You should use mirrored raidz2 sets

Re: ZFS: Use mirror vdevs, not RAIDZ

#17

Post by aidenn0 » Wed, Dec 09, 2020, 3:02 PM UTC

jryb wrote: post #10

My gripe with this is that the author assumes everyone has the same workload and prefers the same set of tradeoffs. I use raidz1 on my home desktop precisely because I would absolutely prefer to have to wait for a resilver than lose data. “So backup your data!” - of course, but that’s just an implicitly larger pool.

TFA suggests using mirrors instead of RAIDZ. There is no case in which RAIDZ1 is more durable against data loss than a mirror is.

Re: ZFS: Use mirror vdevs, not RAIDZ

#18

Post by js2 » Wed, Dec 09, 2020, 3:17 PM UTC

This article is pretty hand-wavy. It doesn't give any empirical numbers at all.

I've had a small FreeNAS server using 4 x 3TB SATA drives in a mirrored config for years now and it's out of space. I'm about to build a new server using used 10 x 3TB SAS drives and intend to put all ten disks into a RAIDZ2 vdev. I care more about space than performance or rebuild times. Before I load it with data, I'll do some testing of read/write performance and rebuilding times. If they're unacceptable, I'll try two smaller RAIDZ vdevs and if that still doesn't work, I'll go back to mirrors.

Re: ZFS: Use mirror vdevs, not RAIDZ

#19

Post by Aaargh20318 » Wed, Dec 09, 2020, 3:17 PM UTC

One problem with mirror vdevs vs. RAIDZ2 : In the RAIDZ2 case you can lose any two drives and still have your data, in the mirror vdev case, 2 drives failing in the same vdev means all your data is gone. You could potentially be okay with losing half your drives, but only if all the failed drives happen to be part of different vdevs.

Re: ZFS: Use mirror vdevs, not RAIDZ

#20

Post by heavenlyblue » Wed, Dec 09, 2020, 3:18 PM UTC

Aaargh20318 wrote: post #19

One problem with mirror vdevs vs. RAIDZ2 : In the RAIDZ2 case you can lose any two drives and still have your data, in the mirror vdev case, 2 drives failing in the same vdev means all your data is gone. You could potentially be okay with losing half your drives, but only if all the failed drives happen to be part of different vdevs.

That's why you can RAIDZ3.