Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

OpenZFS – add disks to existing RAIDZ

github.com

Post reply on HN
Print view View original

141–150 of 177 posts

Re: OpenZFS – add disks to existing RAIDZ

#141

Post by kalleboo »

Lex-2008 wrote: post #113
postmodest wrote: post #88

Earlier quoted context omitted.

You can add devices as vdevs to a pool, though. You can't add disks to a vdev. (IIRC) It's no different to LVM in that aspect.

IIRC, the pool is effectively RAID0 over vdevs: if any single vdev dies - whole pool dies. Do i remember this correctly? So if over the years I got, say, five disks of varying quality - I shouldn't add them to a pool as five individual vdevs :)

Correct. I lied when I said you can't add single disks, I meant you shouldn't add single disks. It's not like e.g. Synology where you can add a single disk and it will expand your underlying RAID5/6 arrays

Re: OpenZFS – add disks to existing RAIDZ

#142

Post by louwrentius »

didntcheck wrote: post #124
louwrentius wrote: post #52

Earlier quoted context omitted.

The key issue is that you basically have to rewrite all existing data to regain that lost 3TB. This takes a huge amount of time and the ZFS developers have decided not to automate this as part of this feature. You can do this yourself though when convenient to get those lost TB back. The RAID VDEV expansion feature actually was quite stale and wasn’t being worked on afaik until this sponsorship.

Was it confirmed anywhere that this space-inefficiency-until-rewrite issue was why it stagnated? I remember looking up the progress a few months ago and being perplexed by the radio silence, given there were celebrations that we were apparently on the home straight back in 2021

No, that’s not what I meant. It’s unrelated. After Matthew Ahrends initial commit years ago basically nothing happened.

The stuff about rewriting data I just wrote to clarify

Re: OpenZFS – add disks to existing RAIDZ

#143

Post by ilyt »

wtallis wrote: post #82
ilyt wrote: post #76

Earlier quoted context omitted.

Eh, it can kinda work. If you have 1, 2,3,4TB drives you can make RAID10 out of first GB, then stitch 2TB RAID0 from second and third and RAID1 it with 2TB out of 4th drive. then raid1 out of remaining GB on 3rd and 4th. Then glue it together with LVM and hope for best.

Or you can just put all those devices into one BTRFS RAID 1 filesystem and not have to worry about it.

BTRFS doesn't exactly had great history with fs bugs

Re: OpenZFS – add disks to existing RAIDZ

#144

Post by ilyt »

edmundsauto wrote: post #73
ilyt wrote: post #72

Earlier quoted context omitted.

....vs stressing all of them for parity rebuild.

Wouldn't it be better to apply stress over more drives to minimize the chances that you lose a second drive during a rebuild?

To replace drive in RAID5/6 array, you need to read the entirety of every drive

To replace drive in RAID1 you need to read the entirety of one drive.

Instead of reading one whole drive worth of stress, you're reading N drives worth of stress

But it is funny that the people making arguments about "stressing drives" don't even fucking know how RAID works...

Re: OpenZFS – add disks to existing RAIDZ

#145

Post by ilyt »

ErneX wrote: post #118
edmundsauto wrote: post #73

Earlier quoted context omitted.

Wouldn't it be better to apply stress over more drives to minimize the chances that you lose a second drive during a rebuild?

I believe this is the article I read when I started: https://jrs-s.net/2015/02/06/zfs-you-should-use-mirror-vdevs...

That talks more about RAID5/Z1 vs mirroring.

You still need 6/Z2 if you want to have reasonable fault tolerance, unless you want to waste ungodly amount of space.

Re: OpenZFS – add disks to existing RAIDZ

#146

Post by edmundsauto »

keeperofdakeys wrote: post #99
edmundsauto wrote: post #74

Earlier quoted context omitted.

Is the amount of data read/written the same? I'm not clear why wall time is the relevant metric, unless that's the critical driver of failure likelihood. I would have guessed it's not time but bytes.

A mirror resilver is a relatively linear and sequential rewrite, so its very fast. Raidz resilvers require lots of random reads and writes across all the drives, and requires waiting for all drives to read data before it can be written to the replaced drive - "herding cats" sounds appropriate here.

Do random reads and writes out more stress on the drive mechanism? I’m wondering if this increase in resliver time adds risk of data loss, which is probably the more important factor than clock time. (But I’m open to learning if I am wrong)

Re: OpenZFS – add disks to existing RAIDZ

#147

Post by _trapexit »

gregmac wrote: post #110
RulerOf wrote: post #104

Earlier quoted context omitted.

I gave snapraid a serious look a few months back and decided it might not be for me because the act of balancing writes out to the "array" member disks appeared to be manual. I didn't want to point applications at 100 T of "free" space only for attires to start blocking after 8. Am I mistaken about that?

Sorry I said unionfs but it's actually handled by mergerfs [1], and it's all automatic. There are a whole boatload of policies [2] to control writes. I use "existing path, least free space". Once a path is created, it keeps using it for new files in that path. If it runs out of space, it creates that same path on another drive. If the path exists on both drives for some reason, my rationale is this keeps most of the…

> If it runs out of space, it creates that same path on another drive.

That's not how it works.

The policy picks what branch to use and then once selected mergerfs will clone the relative path as needed. With "ep" policies it will never select a branch that doesn't have the full relative path. "msp" will always rerun the check one level up in the hierarchy if nothing is found at the current level.

Re: OpenZFS – add disks to existing RAIDZ

#148

Post by cassianoleal »

lyjia wrote: post #111
cassianoleal wrote: post #6

Earlier quoted context omitted.

What do you mean by released? It hasn't even been merged yet. :)

Misread the pull request. But this means it is close to release! It has been in the works for some time.

I do hope so! I have 2 drives waiting to be added to my pool. :)

Re: OpenZFS – add disks to existing RAIDZ

#149

Post by edmundsauto »

ilyt wrote: post #144
edmundsauto wrote: post #73

Earlier quoted context omitted.

Wouldn't it be better to apply stress over more drives to minimize the chances that you lose a second drive during a rebuild?

To replace drive in RAID5/6 array, you need to read the entirety of every drive To replace drive in RAID1 you need to read the entirety of one drive. Instead of reading one whole drive worth of stress, you're reading N drives worth of stress But it is funny that the people making arguments about "stressing drives" don't even fucking know how RAID works...

However, RAID1, you stress the drive that has your only copy of the data. With 5/6 (RAIDz w/ 2 spares as well) you can have multiple copies of the data, so although you are overall increasing stress on your fleet, wouldn't it have a lower probability of data loss?

Drive stress (for me) is mostly a concern about data loss.

Re: OpenZFS – add disks to existing RAIDZ

#150

Post by labawi »

didntcheck wrote: post #125
ilyt wrote: post #72

Earlier quoted context omitted.

....vs stressing all of them for parity rebuild.

However RAIDZ2 can survive up to one of those stressed disks failing, while a 2-mirror has everything riding on that single survivor. It seems intuitive that mirrors would be safer, yeah, but if you run the binomial distribution numbers there are some realistic combinations of array size and individual drive failure probabilities where RAIDZ2 really does appear to be safer Imagine a gambler telling you "you can eithe…

If we were to only compare survival after a fixed number of drives failed vs. storage efficiency, RAIDzN should always come out ahead in any configuration - with mirrors you can get unlucky drives choice fail, with RAIDzN any choice is a good choice. Only way to have RAIDz fail sooner than mirror is to have a comparatively less redundant setup (your choice of N and K).

Realistically though, RAIDz recovery is longer and more stressful, so more of your drives can fail in the critical period, and, assuming you have backups, your storage is there for for usability - mirroring gives you a performant usable system during a fast recovery for the price of a small chance of complete data loss (but you have backups?) vs RAIDz that gives you long recovery pains on a degraded system, but I expect a smaller chance of data loss on a lightly loaded system.

Post reply on HN