Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

ZFS: Use mirror vdevs, not RAIDZ

jrs-s.net

Post reply on HN
Print view View original

71–80 of 133 posts

Re: ZFS: Use mirror vdevs, not RAIDZ

#71

Post by KingMachiavelli »

hedora wrote: post #38
tjoff wrote: post #25

Earlier quoted context omitted.

RAID is not a backup solution, its an uptime solution. I know. Doesn't change the fact there exist no sensible way to backup a decently sized pool in a home-user scenario. Now we might disagree on what is sensible. And some things might not need a backup.

I backup my raid array to the cloud using hyperbackup, but any incremental backup solution would work. It’s ~$60/TB/year: https://www.backblaze.com/b2/cloud-storage-pricing.html I only have a few TB, but the setup cost almost $1000 (it is less than half full), and it will take longer than I expect the hardware to last to spend $1000 on backups.

If you only need backups in case of whole drive failure, AWS S3 glacier is cheaper at $0.004/GB/month.

But it's still $4/month/TB or $48/TB-Year. Assuming a two 10TB drives are $300 each (actually can get them for less than $200) and last 5 years or warranty expires. ($300*2 drives /5 years / 10 TB = $12/TB-Year). So AWS glacier is still ~4x the cost of buying physical drives with RAID 1 and keeping them at a friends house.

Honestly if there is any cloud storage that trades resiliency for price that would be awesome. As long as the cloud storage can tell me if any of the storage has been corrupted/lost, I'd except a 10% yearly failure rate. If my home raid is 95% and the cloud is 90% then I have an effective yearly data loss of only 0.5%.

Re: ZFS: Use mirror vdevs, not RAIDZ

#72

Post by dspillett »

cortesoft wrote: post #30
tjoff wrote: post #25

Earlier quoted context omitted.

RAID is not a backup solution, its an uptime solution. I know. Doesn't change the fact there exist no sensible way to backup a decently sized pool in a home-user scenario. Now we might disagree on what is sensible. And some things might not need a backup.

Isn’t the backup solution to always get two of whatever you build? A primary raid backed up to a second pool?

You need to do more than just keep a backup of current state, if you want to deal with problems other than complete failure/destruction of the primary. What about accidental deletion and incorrect editing (done too long ago for "undo" to work)?

Re: ZFS: Use mirror vdevs, not RAIDZ

#73

Post by Rafuino »

angry_octet wrote: post #27

I would like to see an update for this for SSDs. The devices are black boxes of mystery, possibly likely to fail nearly simultaneously (at least in terms of being unable to write). Early on we left 20% if capacity unformatted to allow more wear leveling, but running them in RAID1 seemed both necessary and likely to cause synchronous errors. RAIDZ1 over three disks might be okay. Obviously huge R/W bandwidth so resilv…

Battery backed flash sounds like a huge PitA to deal with though. Where do you even get that?

Re: ZFS: Use mirror vdevs, not RAIDZ

#74

Post by dnautics »

rsync wrote: post #46
tutfbhuf wrote: post #42

Earlier quoted context omitted.

May I ask what your worst incident with ZFS was ever since 2012?

Some background ... We have a fairly robust drive-proofing procedure prior to deployment - I think we still use 'badblocks' and really beat them up for 5-7 days. It's not enough to merely not fail - we have to see perfect returns on the SMART diagnostics and zero complaints from FreeBSD. So we're starting with a known-good population of drives. Then we monitor them very closely - again with both SMART and FreeBSD/ZFS…

thank you for this.

can you speak to the retention rate post culling using your proofing? And do you have preferred manufacturers?

Also, do you bother to do "the trick" where you zfs across heterogeneous SKUs to reduce the risk of use-case-coupled failures?

Re: ZFS: Use mirror vdevs, not RAIDZ

#75

Post by aidenn0 »

Naac wrote: post #61
aidenn0 wrote: post #17

Earlier quoted context omitted.

TFA suggests using mirrors instead of RAIDZ. There is no case in which RAIDZ1 is more durable against data loss than a mirror is.

Doesn't that ignore the fact that RAIDZ1 gives you more storage space than a mirror?

The comment I was replying to had no mention whatsoever of storage space requirements.

Re: ZFS: Use mirror vdevs, not RAIDZ

#76

Post by res0nat0r »

dsr_ wrote: post #33
tjoff wrote: post #25

Earlier quoted context omitted.

RAID is not a backup solution, its an uptime solution. I know. Doesn't change the fact there exist no sensible way to backup a decently sized pool in a home-user scenario. Now we might disagree on what is sensible. And some things might not need a backup.

zfs send. Keep different filesystems for system backup, personal data, media storage, gross hoarding. Consider not backing up the media storage and gross hoarding filesystems. Targets: 1. That NAS off in the corner built from an old desktop. 2. rsync.net and similar. 3. your encrypted cloud backup service of choice, through an adapter. If the value of your data is less than the cost to protect it, you know it is disp…

Backing up TB's of data isn't cheap no matter what type of cloud backup solution you're picking unfortunately. I don't think most here want to spend $1000/year on backing up their entire NAS.

Re: ZFS: Use mirror vdevs, not RAIDZ

#77

Post by res0nat0r »

KingMachiavelli wrote: post #71
hedora wrote: post #38

Earlier quoted context omitted.

I backup my raid array to the cloud using hyperbackup, but any incremental backup solution would work. It’s ~$60/TB/year: https://www.backblaze.com/b2/cloud-storage-pricing.html I only have a few TB, but the setup cost almost $1000 (it is less than half full), and it will take longer than I expect the hardware to last to spend $1000 on backups.

If you only need backups in case of whole drive failure, AWS S3 glacier is cheaper at $0.004/GB/month. But it's still $4/month/TB or $48/TB-Year. Assuming a two 10TB drives are $300 each (actually can get them for less than $200) and last 5 years or warranty expires. ($300*2 drives /5 years / 10 TB = $12/TB-Year). So AWS glacier is still ~4x the cost of buying physical drives with RAID 1 and keeping them at a friends…

Deep Archive is my fav backup solution to just store a lot of data cheaply as there isn't really anything better, just the AWS egress charge is really the only thing they are keeping at a high price and for sure is for lock-in. .09c/GB is still too high to restore TB's of data if you need a full NAS rebuild and need to re-download everything.

Re: ZFS: Use mirror vdevs, not RAIDZ

#78

Post by labawi »

nix23 wrote: post #12
geerlingguy wrote: post #9

Every time I make this argument (whether regular RAID or ZFS), everyone pulls out their pitchforks and tells me about how they have run RAID 5 or 6 for years and never had a problem, plus "I can have two drives fail! You would lose your array if the wrong two drives fail!" But drive failure is much more likely during an operation like a resilver/resync, and I wonder if the majority of people espousing more risky RAID…

I then tell a real horror-story about my past, not so long ago there was a EVA (A SAN from HP) with a freaking out hardware Raid out of nowhere (uptime about 200 days no Firmware update before nothing) it just started to freak out, the story goes on but the lesson...No more HW-Raid/proprietary stuff for me EVER again.

While none for me either, I used to look down on HW RAID etc. Lately I've realized, such solutions are genuinely useful and more likely to survive/work properly, if the people managing it do not understand the technology, don't want to understand or just don't have the time for it.

HW RAID with red blinking lights seems more likely to survive than a SW RAID that works until it doesn't, because you lost all the redundancy and noone there knows how either work.

Re: ZFS: Use mirror vdevs, not RAIDZ

#79

Post by ubercow13 »

mulmen wrote: post #57
tjoff wrote: post #22

I don't agree. And if one of your disks failed, and age was a factor… you’re going to be sweating bullets wondering if another will fail before your resilver completes. So every every single time you loose any drive in a mirror setup you risk all the data on all drives in the entire pool. I sure do hope you aren't on vacation and/or have to order a drive online. It all depends on your use case. For me, raidz3 wins ea…

What’s wrong with rsync.net or tarsnap for backups?

They are extremely expensive.

Re: ZFS: Use mirror vdevs, not RAIDZ

#80

Post by tracker1 »

tjoff wrote: post #25
necheffa wrote: post #24

Earlier quoted context omitted.

Repeat after me: RAID is not a backup solution, its an uptime solution.

RAID is not a backup solution, its an uptime solution. I know. Doesn't change the fact there exist no sensible way to backup a decently sized pool in a home-user scenario. Now we might disagree on what is sensible. And some things might not need a backup.

That said, it reduces risk of loss when an actual full backup solution is impractical... I have about 10TB of media on my current 12tb (4x4 raid5) nas box. My next plan is a 6-drive raid z2 with 8-12tb drives early next year. I can't justify doubling the cost or more to ensure it's all backed up. The important bits (to me) are in various redundant places... that doesn't mean I want to lose it and the parity options allow for some safety without too much lose of storage space.
Post reply on HN