Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

OpenZFS – add disks to existing RAIDZ

github.com

Post reply on HN
Print view View original

41–50 of 177 posts

Re: OpenZFS – add disks to existing RAIDZ

#41

Post by mrighele »

hinkley wrote: post #36

I’m frustrated because this feature was mentioned by Schwartz when it was still in beta. I thought a new era of home computing was about to start. It didn’t, and instead we got The Cloud, which feels like decentralization but is in fact massive centralization (organizational, rather than geographical). Some of us think people should be hosting stuff from home, accessible from their mobile devices. But the first and t…

> I’m a developer, I can afford it. None of my friends can. Mom definitely can’t.

The only thing that can work for your mom and your friend is, in my opinion, a pair of disks in mirror. When the space finished, buy another box with two other disk in mirror. Anything more than this is not only too complex for the average user but also too expensive.

Re: OpenZFS – add disks to existing RAIDZ

#42

Post by mustache_kimono »

crote wrote: post #29

> After the expansion completes, old blocks remain with their old data-to-parity ratio (e.g. 5-wide RAIDZ2, has 3 data to 2 parity), but distributed among the larger set of disks. New blocks will be written with the new data-to-parity ratio (e.g. a 5-wide RAIDZ2 which has been expanded once to 6-wide, has 4 data to 2 parity). Does anyone know why this is the case? When expanding an array which is getting full this wi…

> Does anyone know why this is the case? > Considering the process is already reading and rewriting basically the entire array, why not recalculate the parity as well?

IANA expert but my guess is -- because, here, you don't have to modify block pointers, etc.

ZFS RAIDZ is not like traditional RAID, as it's not just a sequence of arbitrary bits, data plus parity. RAIDZ stripe width is variable/dynamic, written in blocks (imagine a 128K block, compressed to ~88K), and there is no way to quickly tell where the parity data is within a written block, where the end of any written block is, etc.

If you had to instead, modify the block pointers, I'd assume you have to also change each block in the live tree and all dependent (including snapshot) blocks at the same time? That sounds extraordinarily complicated (and this is the data integrity FS!), and much slower, than just blasting through the data, in order.

To do what you want, you can do what one could always do -- zfs send/recv between a filesystem between and old and new filesystem.

Re: OpenZFS – add disks to existing RAIDZ

#43

Post by wtallis »

Dwedit wrote: post #39

So is it safe to use btrfs for a basic Raid-1 yet?

Yeah, the non-parity RAID modes have been safe for a pretty long time, as long as you RTFM when something goes wrong instead of assuming the recovery procedures match what you'd expect coming from a background of traditional RAID or how ZFS does it. I've been using RAID-1 (with RAID1c3 for metadata since that feature became available) on a NAS for over a decade now without loss of data despite loss of more drives over the years than the array started out with.

Re: OpenZFS – add disks to existing RAIDZ

#44

Post by hinkley »

mrighele wrote: post #41
hinkley wrote: post #36

I’m frustrated because this feature was mentioned by Schwartz when it was still in beta. I thought a new era of home computing was about to start. It didn’t, and instead we got The Cloud, which feels like decentralization but is in fact massive centralization (organizational, rather than geographical). Some of us think people should be hosting stuff from home, accessible from their mobile devices. But the first and t…

> I’m a developer, I can afford it. None of my friends can. Mom definitely can’t. The only thing that can work for your mom and your friend is, in my opinion, a pair of disks in mirror. When the space finished, buy another box with two other disk in mirror. Anything more than this is not only too complex for the average user but also too expensive.

Keeping track of a heterogenous drive array is just as big an imposition.

Re: OpenZFS – add disks to existing RAIDZ

#45

Post by canvascritic »

this is a really neat addition to raid-z. i recall setting up my zfs pool in the early 2000s and grappling with disk counts because of how rigid expansion was. Good times. this would've made things so much simpler. small nitpick: in the "during expansion" bit, I thought he could have elaborated a touch on restoring the "health of the raidz vdev" part, didn't really follow his reasoning there. but overall, looking forward to this update. nice work.

Re: OpenZFS – add disks to existing RAIDZ

#46

Post by magicalhippo »

crote wrote: post #29

> After the expansion completes, old blocks remain with their old data-to-parity ratio (e.g. 5-wide RAIDZ2, has 3 data to 2 parity), but distributed among the larger set of disks. New blocks will be written with the new data-to-parity ratio (e.g. a 5-wide RAIDZ2 which has been expanded once to 6-wide, has 4 data to 2 parity). Does anyone know why this is the case? When expanding an array which is getting full this wi…

> Considering the process is already reading and rewriting basically the entire array, why not recalculate the parity as well?

Because snapshots might refer to the old blocks. Sure you could recompute, but then any snapshots would mean those old blocks would have to stay around so now you've taken up ~twice the space.

Re: OpenZFS – add disks to existing RAIDZ

#47

Post by Quekid5 »

mrighele wrote: post #41
hinkley wrote: post #36

I’m frustrated because this feature was mentioned by Schwartz when it was still in beta. I thought a new era of home computing was about to start. It didn’t, and instead we got The Cloud, which feels like decentralization but is in fact massive centralization (organizational, rather than geographical). Some of us think people should be hosting stuff from home, accessible from their mobile devices. But the first and t…

> I’m a developer, I can afford it. None of my friends can. Mom definitely can’t. The only thing that can work for your mom and your friend is, in my opinion, a pair of disks in mirror. When the space finished, buy another box with two other disk in mirror. Anything more than this is not only too complex for the average user but also too expensive.

I think I would advise against a direct mirroring -- instead, I'd do sync-every-24-hours or something similar.

Both schemes are vulnerable to the (admittedly rarer) errors where both drives fail simultaneously (e.g. mobo fried them) or are just ... destroyed by a fire or whatever.

A periodic sync (while harder to set up) will occasionally save you from the deleting the wrong files which mirroring doesn't.

Either way: Any truly important data (family photos/videos, etc.) needs to be saved periodically to remote storage. There's no getting around that if you really care about the data.

Re: OpenZFS – add disks to existing RAIDZ

#48

Post by gregmac »

hinkley wrote: post #36

I’m frustrated because this feature was mentioned by Schwartz when it was still in beta. I thought a new era of home computing was about to start. It didn’t, and instead we got The Cloud, which feels like decentralization but is in fact massive centralization (organizational, rather than geographical). Some of us think people should be hosting stuff from home, accessible from their mobile devices. But the first and t…

I can afford it, but have a hard time justifying the costs, not to mention scrapped (working) hardware and inconvenience (of swapping to a whole new array).

I started using snapraid [1] several years ago, after finding zfs couldn't expand. Often when I went to add space the "sweet spot" disk size (best $/TB) was 2-3x the size of the previous biggest disk I ran. This was very economical compared to replacing the whole array every couple years.

It works by having "data" and "parity" drives. Data drives are totally normal filesystems, and joined with unionfs. In fact you can mount them independently and access whatever files are on it. Parity drives are just a big file that snapraid updates nightly.

The big downside is it's not realtime redundant: you can lose a day's worth of data from a (data) drive failure. For my use case this is acceptable.

A huge upside is rebuilds are fairly painless. Rebuilding a parity drive has zero downtime, just degraded performance. Rebuilding a data drive leaves it offline, but the rest work fine (I think the individual files are actually accessible as they're restored though). In the worst case you can mount each data drive independently on any system and recover its contents.

I've been running the "same" array for a decade, but at this point every disk has been swapped out at least once (for a larger one), and it's been in at least two different host systems.

[1] https://www.snapraid.it/

Re: OpenZFS – add disks to existing RAIDZ

#49

Post by ErneX »

I’m not an expert whatsoever but what I’ve been doing for my NAS is using mirrored VDEVs. Started with one and later on added a couple more drives for a second mirror.

Coincidentally one of the drives of my 1st mirror died few days ago after rebooting the host machine for updates and I replaced it today, it’s been resilvering for a while.

Post reply on HN