Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

OpenZFS – add disks to existing RAIDZ

github.com

Post reply on HN
Print view View original

121–130 of 177 posts

Re: OpenZFS – add disks to existing RAIDZ

#121

Post by voytec »

jedberg wrote: post #30

I wish Apple and Oracle would have just sorted things out and made ZFS the main filesystem for the Mac. Way back in the day when they first designed Time Machine, it was supposed to just be a GUI for ZFS snapshots. How cool would it be if we had a great GUI for ZFS (snapshots, volume management, etc.). I could buy a new external disk, add it to a pool, have seamless storage expansion. It would be great. Ah, what coul…

> How cool would it be if we had a great GUI for ZFS

Not in workstation OS, maybe not "great" but tools like FreeNAS had ZFS GUI sorted out[1] pretty decently and user-friendly for some time. I've seen storage laymen set these up, learn on the go and not screw up.

[1] https://windows-cdn.softpedia.com/screenshots/FreeNAS_1.jpg

[1] https://www.ixsystems.com/documentation/freenas/9.3/freenas_...

Re: OpenZFS – add disks to existing RAIDZ

#122

Post by gliaskos »

ksec wrote: post #100
Terretta wrote: post #59

Earlier quoted context omitted.

> How cool would it be if we had a great GUI for ZFS (snapshots, volume management, etc.). I could buy a new external disk, add it to a pool, have seamless storage expansion. See QNAP HERO 5: https://www.qnap.com/static/landing/2021/quts-hero-5.0/en/in... NAS Options: https://www.qnap.com/en-us/product/?conditions=4-3 // This was about MacOS, and yes, that would be cool. But QNAP is remarkably MacOS-friendly, includi…

If you follow Qnap you may be aware of their security nightmare in recent years. And HERO OS is only available on the Enterprise range of NAS, not even Prosumer class NAS.

Something like TVS-h474 seems quite nice for a wide range of uses.

I have TVS-h1288X and I am really happy with it, maybe it's a bit overkill for most.

That said, I could not use zfs send from my prev fbsd server, there are some compat checks in QNAP's version. Used nfs to migrate from my decade++ old pool.

Using Tailscale for access, they have QNAP builds on GH.

Re: OpenZFS – add disks to existing RAIDZ

#123

Post by didntcheck »

katbyte wrote: post #69
crote wrote: post #29

> After the expansion completes, old blocks remain with their old data-to-parity ratio (e.g. 5-wide RAIDZ2, has 3 data to 2 parity), but distributed among the larger set of disks. New blocks will be written with the new data-to-parity ratio (e.g. a 5-wide RAIDZ2 which has been expanded once to 6-wide, has 4 data to 2 parity). Does anyone know why this is the case? When expanding an array which is getting full this wi…

I imagine there would eventually be a way / option to automatically rewrite old blocks. Or at least I would hope so because usually when your adding new disks the array is going to be near full

IIRC one of the developers was asked about this and said it would be about as much additional effort as the whole expansion feature so far. There is of course the user-level workaround of (roughly) `for file in *; do cp $file $file.new && mv $file.new $file && done`, but if I understand correctly you'd need to do this with no snapshots, otherwise the space from $file wouldn't be freed so you'd need an array that's at least half empty

Re: OpenZFS – add disks to existing RAIDZ

#124

Post by didntcheck »

louwrentius wrote: post #52
crote wrote: post #29

> After the expansion completes, old blocks remain with their old data-to-parity ratio (e.g. 5-wide RAIDZ2, has 3 data to 2 parity), but distributed among the larger set of disks. New blocks will be written with the new data-to-parity ratio (e.g. a 5-wide RAIDZ2 which has been expanded once to 6-wide, has 4 data to 2 parity). Does anyone know why this is the case? When expanding an array which is getting full this wi…

The key issue is that you basically have to rewrite all existing data to regain that lost 3TB. This takes a huge amount of time and the ZFS developers have decided not to automate this as part of this feature. You can do this yourself though when convenient to get those lost TB back. The RAID VDEV expansion feature actually was quite stale and wasn’t being worked on afaik until this sponsorship.

Was it confirmed anywhere that this space-inefficiency-until-rewrite issue was why it stagnated? I remember looking up the progress a few months ago and being perplexed by the radio silence, given there were celebrations that we were apparently on the home straight back in 2021

Re: OpenZFS – add disks to existing RAIDZ

#125

Post by didntcheck »

ilyt wrote: post #72
edmundsauto wrote: post #57

Earlier quoted context omitted.

I’ve read this is suboptimal because you are now stressing the drive that has the only copy of your data to rebuild. what are your thoughts?

....vs stressing all of them for parity rebuild.

However RAIDZ2 can survive up to one of those stressed disks failing, while a 2-mirror has everything riding on that single survivor. It seems intuitive that mirrors would be safer, yeah, but if you run the binomial distribution numbers there are some realistic combinations of array size and individual drive failure probabilities where RAIDZ2 really does appear to be safer

Imagine a gambler telling you "you can either throw one k-faced die and you lose if it comes up on 1, or throw n k-faced dice and you lose if two come up on 1". Depending on k and n, the second really can be the better choice

Often people cite the faster rebuild of mirrors as a safety advantage, but the same amount of rebuild IO will occur regardless of how long it takes. Yes, there will be more non-rebuild IO in a larger time window, but unless that routine load was causing disks to fail weekly then I doubt it will change the numbers non-negligibly. It will of course affect array performance though, so mirrors for performance is a good argument

Re: OpenZFS – add disks to existing RAIDZ

#126

Post by PlutoIsAPlanet »

keeperofdakeys wrote: post #108
wtallis wrote: post #103

Earlier quoted context omitted.

> The BTRFS raid1 feature is very badly named, You're going to have to support that claim a bit better. The core idea of RAID 1 is mirroring data, which BTRFS RAID 1 mode definitely does. Striping is not an essential part of RAID 1 (hence RAID 10), and reading data from two disks in parallel is an optional performance optimization that is not performed by all RAID 1 implementations (but could be implemented for BTRFS…

RAID1 is about mirroring disks, BTRFS RAID1 mirrors block groups. Plus traditionally a RAID1 of 3 disks will mirror the same data on all disks, which is different to how the RAID1 mode on BTRFS acts. So the name leads to misunderstandings since it doesn't act like RAID1 at all. It'd be way easier to talk about if it had a unique name, and you could say "It's like RAID1". Despite all that I do like the mode, and use i…

> Plus traditionally a RAID1 of 3 disks will mirror the same data on all disks, which is different to how the RAID1 mode on BTRFS acts.

You have raid1c mode in BTRFS which does this.

Re: OpenZFS – add disks to existing RAIDZ

#127

Post by cromka »

gjvc wrote: post #120
xoa wrote: post #56

Earlier quoted context omitted.

Yes, this will always depress me, and to me personally be one of the ultimate evils of the court invented idea of "software patents". I've used ZFS with Macs sicne 2011, but without Apple onboard it's never been as smooth as it should have been and has gotten more difficult in some respects. There was a small window where we might have had a really universal, really solid FS with great data guarantees and features. A…

1000x this. A universal filesystem for UNIX would have been a wonderful thing, and would have been the best legacy Sun could have left.

Would become so ubiquitous that I can see how Microsoft would even eventually add a native support for it, especially in the WSL era.

Re: OpenZFS – add disks to existing RAIDZ

#128

Post by cromka »

toast0 wrote: post #92
ilyt wrote: post #78

Earlier quoted context omitted.

So, why OpenZFS can't fix their license ?

OpenZFS is a fork of the Sun/Oracle release of ZFS. The OpenZFS maintainers cannot change the license without consent of the copyright owners, and Oracle is unlikely to consent. Oracle could potentially change the terms in a future release of CDDL, and that might work, too. Similarly, Linux probably has too many contributors, some of which are no longer living, to come to an agreement on a fixed license either. Altho…

> OpenSSL did one recently

VLC did, too, although with probably even less contributors.

Re: OpenZFS – add disks to existing RAIDZ

#129

Post by didntcheck »

keeperofdakeys wrote: post #99
edmundsauto wrote: post #74

Earlier quoted context omitted.

Is the amount of data read/written the same? I'm not clear why wall time is the relevant metric, unless that's the critical driver of failure likelihood. I would have guessed it's not time but bytes.

A mirror resilver is a relatively linear and sequential rewrite, so its very fast. Raidz resilvers require lots of random reads and writes across all the drives, and requires waiting for all drives to read data before it can be written to the replaced drive - "herding cats" sounds appropriate here.

That makes sense. Usually the received wisdom I hear is "RAIDZ is slower due to the parity calculations", which has always seemed dubious to me on modern CPUs, given we can do much more complex compression and encryption at way faster then disk speed (for HDDs, at least)

Re: OpenZFS – add disks to existing RAIDZ

#130

Post by didntcheck »

gregmac wrote: post #48
hinkley wrote: post #36

I’m frustrated because this feature was mentioned by Schwartz when it was still in beta. I thought a new era of home computing was about to start. It didn’t, and instead we got The Cloud, which feels like decentralization but is in fact massive centralization (organizational, rather than geographical). Some of us think people should be hosting stuff from home, accessible from their mobile devices. But the first and t…

I can afford it, but have a hard time justifying the costs, not to mention scrapped (working) hardware and inconvenience (of swapping to a whole new array). I started using snapraid [1] several years ago, after finding zfs couldn't expand. Often when I went to add space the "sweet spot" disk size (best $/TB) was 2-3x the size of the previous biggest disk I ran. This was very economical compared to replacing the whole…

I like Snapraid, but there are a few downsides worth mentioning

* Rebuilds are semi-offline, as you said. Almost every other solution, even Unraid, will immediately emulate the data from a failed drive. On Snapraid you have to wait for each file to be restored, and depending on your union setup this may mean you have directories with half the files missing

* Whenever you modify or delete a file between syncs, your parity is now out of sync. This means some other files may fail to restore if you have a failure now. However using more than single parity will make this far less likely to happen. Another way to fix this completely the "snapraid-btrfs" tool, which runs Snapraid on snapshots of independent btrfs disks (somewhat like Synology?), meaning the old data is still available

* It saves almost no file metadata, not even owner and mode. Restored files just use the umask of the user running the restore command. A minor one, but surprisingly annoying

However one big advantage over Unraid it has is how transactional and rigorous it is. A power failure can cause Unraid parity to desync, with no clear way to know which disk is right. Snapraid OTOH is designed to survive interruptions gracefully, and checksums all files so should never accidentally restore corrupt data. And can detect silent drive failure ("bitrot") as a bonus

And for a typical home NAS storing movies and family photos (mostly append-only), those downsides are probably no big deal anyway

Post reply on HN