Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

OpenZFS – add disks to existing RAIDZ

github.com

Post reply on HN
Print view View original

71–80 of 177 posts

Re: OpenZFS – add disks to existing RAIDZ

#71

Post by ilyt »

allanjude wrote: post #65
crote wrote: post #29

> After the expansion completes, old blocks remain with their old data-to-parity ratio (e.g. 5-wide RAIDZ2, has 3 data to 2 parity), but distributed among the larger set of disks. New blocks will be written with the new data-to-parity ratio (e.g. a 5-wide RAIDZ2 which has been expanded once to 6-wide, has 4 data to 2 parity). Does anyone know why this is the case? When expanding an array which is getting full this wi…

That is not how this will work. The reason the parity ratio stays the same, is that all of the references to the data are by DVA (Data Virtual Address, effectively the LBA within the RAID-Z vdev). So the data will occupy the same amount of space and parity as it did before. All stripes in RAID-Z are dynamic, so if your stripe is 5 wide and your array is 6 wide, the 2nd stripe will start on the last disk and wrap arou…

> So the data will occupy the same amount of space and parity as it did before.

So you lose data capacity compared to "dumb" RAID6 on mdadm.

If you expand RAID6 from 4+2 to 5+2, you go from using 33.3% data for parity to 28.5% on parity

If you expand RAIDZ from 4+2 to 5+2, your new data will use 28.5%, but your old (which is majority, because if it wasn't you wouldn't be expanding) would still use 33.3% on parity.

Re: OpenZFS – add disks to existing RAIDZ

#72

Post by ilyt »

edmundsauto wrote: post #57
ErneX wrote: post #49

I’m not an expert whatsoever but what I’ve been doing for my NAS is using mirrored VDEVs. Started with one and later on added a couple more drives for a second mirror. Coincidentally one of the drives of my 1st mirror died few days ago after rebooting the host machine for updates and I replaced it today, it’s been resilvering for a while.

I’ve read this is suboptimal because you are now stressing the drive that has the only copy of your data to rebuild. what are your thoughts?

....vs stressing all of them for parity rebuild.

Re: OpenZFS – add disks to existing RAIDZ

#73

Post by edmundsauto »

ilyt wrote: post #72
edmundsauto wrote: post #57

Earlier quoted context omitted.

I’ve read this is suboptimal because you are now stressing the drive that has the only copy of your data to rebuild. what are your thoughts?

....vs stressing all of them for parity rebuild.

Wouldn't it be better to apply stress over more drives to minimize the chances that you lose a second drive during a rebuild?

Re: OpenZFS – add disks to existing RAIDZ

#74

Post by edmundsauto »

deadbunny wrote: post #64
edmundsauto wrote: post #57

Earlier quoted context omitted.

I’ve read this is suboptimal because you are now stressing the drive that has the only copy of your data to rebuild. what are your thoughts?

Mirrored vdevs resilver a lot faster than zX vdevs. Much less chance of the remaining drive dying during a resilver if it takes hours rather than days.

Is the amount of data read/written the same? I'm not clear why wall time is the relevant metric, unless that's the critical driver of failure likelihood. I would have guessed it's not time but bytes.

Re: OpenZFS – add disks to existing RAIDZ

#75

Post by ilyt »

hinkley wrote: post #36

I’m frustrated because this feature was mentioned by Schwartz when it was still in beta. I thought a new era of home computing was about to start. It didn’t, and instead we got The Cloud, which feels like decentralization but is in fact massive centralization (organizational, rather than geographical). Some of us think people should be hosting stuff from home, accessible from their mobile devices. But the first and t…

Unraid "solves" that althought needs some user knowledge (IIRC parity drives must be the biggest one in array and for some reason that isn't automatic)

Ceph actually works quite well for that, althought obviously far more complex than anything for home use. You just tell it "x chunks with N parity" and it will spread it over available drives. Just need more drives than x + n and not too egregious size differences.

> Weighted consistent hashing can handle disparate resources, by assigning more buckets to faster or larger machines. And it can grow and shrink (in a drive array, the two are sequential or simultaneous).

It would need to be more complex than that. Putting chunks on say 2, 6, 8, 12, 22 TB (which might be what you'd get if you just buy "cheapest GB/$" for your NAS over last 10 years!) is more complex than that.

Re: OpenZFS – add disks to existing RAIDZ

#76

Post by ilyt »

hinkley wrote: post #70
wtallis wrote: post #67

Earlier quoted context omitted.

> which rules out all RAID levels except 0, which is nuts. RAID 1, you mean? Because that way you still have a complete copy of your data if one drive fails. BTRFS is excellent for the use case of a wide variety of mismatched drives, because it supports adding and removing drives and rebalancing the array. But for the moment only the RAID 1 modes are really trustworthy. I have a NAS consisting of drives whose adverti…

Raid 1 does not work with heterogenous disk arrays.

Eh, it can kinda work. If you have 1, 2,3,4TB drives you can make RAID10 out of first GB, then stitch 2TB RAID0 from second and third and RAID1 it with 2TB out of 4th drive. then raid1 out of remaining GB on 3rd and 4th.

Then glue it together with LVM and hope for best.

Re: OpenZFS – add disks to existing RAIDZ

#77

Post by ilyt »

Osiris wrote: post #40
hinkley wrote: post #36

I’m frustrated because this feature was mentioned by Schwartz when it was still in beta. I thought a new era of home computing was about to start. It didn’t, and instead we got The Cloud, which feels like decentralization but is in fact massive centralization (organizational, rather than geographical). Some of us think people should be hosting stuff from home, accessible from their mobile devices. But the first and t…

I completely agree. To build my array I had to buy several drives at the same time. To expand I had to buy a new drive, move the data onto the array, and then I’m left with the extra drive I had too buy to temporarily store the data because I can’t add it to the array. I would love to have more options for expandable redundancy.

I just...leave it then create new RAID out of free space once I get more than 2-3 disks that are bigger. But yeah, PITA.

Re: OpenZFS – add disks to existing RAIDZ

#78

Post by ilyt »

tux3 wrote: post #20
Nextgrid wrote: post #16

Earlier quoted context omitted.

> The last few kerfuffles around symbols used by the out-of-tree module laid out the position rather unambiguously. Source, for someone who isn't following kernel mailing lists?

I was thinking of this thread from 5.0 in particular (2019, time flies!) https://lore.kernel.org/all/20190110182413.GA6932@kroah.com/

So, why OpenZFS can't fix their license ?

Re: OpenZFS – add disks to existing RAIDZ

#79

Post by Tijdreiziger »

hinkley wrote: post #36

I’m frustrated because this feature was mentioned by Schwartz when it was still in beta. I thought a new era of home computing was about to start. It didn’t, and instead we got The Cloud, which feels like decentralization but is in fact massive centralization (organizational, rather than geographical). Some of us think people should be hosting stuff from home, accessible from their mobile devices. But the first and t…

https://kb.synology.com/en-global/DSM/tutorial/What_is_Synol...

Re: OpenZFS – add disks to existing RAIDZ

#80

Post by wtallis »

hinkley wrote: post #70
wtallis wrote: post #67

Earlier quoted context omitted.

> which rules out all RAID levels except 0, which is nuts. RAID 1, you mean? Because that way you still have a complete copy of your data if one drive fails. BTRFS is excellent for the use case of a wide variety of mismatched drives, because it supports adding and removing drives and rebalancing the array. But for the moment only the RAID 1 modes are really trustworthy. I have a NAS consisting of drives whose adverti…

Raid 1 does not work with heterogenous disk arrays.

BTRFS RAID 1 works just fine with heterogeneous disks with no extra configuration required; I would not have been able to provide examples from experience otherwise. When writing new data, it allocates two blocks on two separate devices, preferring to allocate space on whatever devices have the most free space. This may not be optimal from a performance perspective, but it does mean there's minimal wasted capacity in most configurations. (The RAID1c3 and RAID1c4 modes work the same, but with three or four copies of each block, all on separate devices.)

Were you assuming that BTRFS RAID 1 meant every block of data gets mirrored across every device? I've seen that assumption before, from people who don't realize there are two ways to generalize RAID1 to more than two devices.

Post reply on HN