Live data from Hacker News

ZFS: Use mirror vdevs, not RAIDZ

jrs-s.net

31–40 of 133 posts

Re: ZFS: Use mirror vdevs, not RAIDZ

#31
post #18

This article is pretty hand-wavy. It doesn't give any empirical numbers at all. I've had a small FreeNAS server using 4 x 3TB SATA drives in a mirrored config for years now and it's out of space. I'm about to build a new server using used 10 x 3TB SAS drives and intend to put all ten disks into a RAIDZ2 vdev. I care more about space than performance or rebuild times. Before I load it with data, I'll do some testing o…

The article is another entry in a long series of bad ZFS articles.

For some reason a lot of people get to a point where they're comfortable with it and suddenly their use case is everyone's, they've become an expert, and you should Just Do What They Say.

I highly recommend people ignore articles like this. ZFS is very flexible, and can serve a variety of workloads. It also assumes you know what you're doing, and the tradeoffs are not always apparent up-front.

If you want to become comfortable-enough with ZFS to make your own choices, I recommend standing up your ZFS box well before you need it, and play with it. Set up configs you'd never use in production, just to see what happens. Yank a disk, figure out how to recover. If you have time, fill it up with garbage and look for yourself how fragmentation effects resilver times. If you're a serious user, join the mailing lists - it is high-signal.

And value random articles on the interwebs telling you the Real Way at what they cost you.

I'm convinced articles like this are a big part of what gives ZFS a bad name. People follow authoritative-sounding bad advice and blame their results on the file system.

Re: ZFS: Use mirror vdevs, not RAIDZ

#32
post #25

Earlier quoted context omitted.

RAID is not a backup solution, its an uptime solution. I know. Doesn't change the fact there exist no sensible way to backup a decently sized pool in a home-user scenario. Now we might disagree on what is sensible. And some things might not need a backup.

Isn’t the backup solution to always get two of whatever you build? A primary raid backed up to a second pool?

Not everyone has that kind of cash lying around, especially if the pool is large and the data is not critical.

Re: ZFS: Use mirror vdevs, not RAIDZ

#33
post #25

Earlier quoted context omitted.

Repeat after me: RAID is not a backup solution, its an uptime solution.

RAID is not a backup solution, its an uptime solution. I know. Doesn't change the fact there exist no sensible way to backup a decently sized pool in a home-user scenario. Now we might disagree on what is sensible. And some things might not need a backup.

zfs send. Keep different filesystems for system backup, personal data, media storage, gross hoarding. Consider not backing up the media storage and gross hoarding filesystems.

Targets:

1. That NAS off in the corner built from an old desktop.

2. rsync.net and similar.

3. your encrypted cloud backup service of choice, through an adapter.

If the value of your data is less than the cost to protect it, you know it is disposable, you just wish it weren't.

Re: ZFS: Use mirror vdevs, not RAIDZ

#34
post #22

I don't agree. And if one of your disks failed, and age was a factor… you’re going to be sweating bullets wondering if another will fail before your resilver completes. So every every single time you loose any drive in a mirror setup you risk all the data on all drives in the entire pool. I sure do hope you aren't on vacation and/or have to order a drive online. It all depends on your use case. For me, raidz3 wins ea…

Repeat after me: RAID is not a backup solution, its an uptime solution.

Nothing in gp's post even remotely indicated that they thought it was

Re: ZFS: Use mirror vdevs, not RAIDZ

#35
post #22

I don't agree. And if one of your disks failed, and age was a factor… you’re going to be sweating bullets wondering if another will fail before your resilver completes. So every every single time you loose any drive in a mirror setup you risk all the data on all drives in the entire pool. I sure do hope you aren't on vacation and/or have to order a drive online. It all depends on your use case. For me, raidz3 wins ea…

> Which is going to hurt since ZFS with buying everything up front is VASTLY more expensive than RAID where you can grow the array as needed - under the assumption that the storage needs grow slowly, which they typically do for home users.

Not with mirrored vdevs, which you can expand pair by pair.

I'm not arguing for the article, just pointing out your error. The article is very flawed and is bad advice. Not because it's bad per se, but the reasoning fails at basic mathematical analysis. There certainly are valid reasons to use mirrored vdevs. One might be, so you can more easily expand your pool incrementally.

Anyway, from personal experience, my thought is that home users should give up on the idea of "incremental expansion" anyway. Double up each time. I'm surprised that you want to say home users aren't going to use 8 drive setups, while at the same time decrying mirrors in favor of parity. If you have a low drive count, the storage efficiency of parity puts you close to that of mirrors anyway. raidz2 with 4 drives as you suggest makes almost no sense vs mirrors.

Re: ZFS: Use mirror vdevs, not RAIDZ

#37
We (rsync.net) have several PB of raidz3 deployed all over the world.

We use conservatively sized (12-15 drive) vdevs and typically join 3 or 4 of those together to make a pool.

I can see getting nervous about raidz2 (sort of analogous to "raid6") after a drive failure ... but losing 4 drives out of 12 in a single raidz3 failure cascade is extremely improbable.

We all sleep quite well with this arrangement and have since we first migrated from UFS2 to ZFS in 2012.

Re: ZFS: Use mirror vdevs, not RAIDZ

#38
post #25

Earlier quoted context omitted.

Repeat after me: RAID is not a backup solution, its an uptime solution.

RAID is not a backup solution, its an uptime solution. I know. Doesn't change the fact there exist no sensible way to backup a decently sized pool in a home-user scenario. Now we might disagree on what is sensible. And some things might not need a backup.

I backup my raid array to the cloud using hyperbackup, but any incremental backup solution would work. It’s ~$60/TB/year:

https://www.backblaze.com/b2/cloud-storage-pricing.html

I only have a few TB, but the setup cost almost $1000 (it is less than half full), and it will take longer than I expect the hardware to last to spend $1000 on backups.

Re: ZFS: Use mirror vdevs, not RAIDZ

#39
post #33
post #25

Earlier quoted context omitted.

RAID is not a backup solution, its an uptime solution. I know. Doesn't change the fact there exist no sensible way to backup a decently sized pool in a home-user scenario. Now we might disagree on what is sensible. And some things might not need a backup.

zfs send. Keep different filesystems for system backup, personal data, media storage, gross hoarding. Consider not backing up the media storage and gross hoarding filesystems. Targets: 1. That NAS off in the corner built from an old desktop. 2. rsync.net and similar. 3. your encrypted cloud backup service of choice, through an adapter. If the value of your data is less than the cost to protect it, you know it is disp…

Yeah as much as my daughter likes the Trolls movie, I'm not paying to back that up.

Re: ZFS: Use mirror vdevs, not RAIDZ

#40
post #22

I don't agree. And if one of your disks failed, and age was a factor… you’re going to be sweating bullets wondering if another will fail before your resilver completes. So every every single time you loose any drive in a mirror setup you risk all the data on all drives in the entire pool. I sure do hope you aren't on vacation and/or have to order a drive online. It all depends on your use case. For me, raidz3 wins ea…

You can add mirrors to increase the pool size. More RAM will speed up the pool, but cache drives also works really well. And you can have a hot spare.
Post reply on HN