Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

OpenZFS – add disks to existing RAIDZ

github.com

Post reply on HN
Print view View original

101–110 of 177 posts

Re: OpenZFS – add disks to existing RAIDZ

#101

Post by keeperofdakeys »

wtallis wrote: post #67
hinkley wrote: post #36

I’m frustrated because this feature was mentioned by Schwartz when it was still in beta. I thought a new era of home computing was about to start. It didn’t, and instead we got The Cloud, which feels like decentralization but is in fact massive centralization (organizational, rather than geographical). Some of us think people should be hosting stuff from home, accessible from their mobile devices. But the first and t…

> which rules out all RAID levels except 0, which is nuts. RAID 1, you mean? Because that way you still have a complete copy of your data if one drive fails. BTRFS is excellent for the use case of a wide variety of mismatched drives, because it supports adding and removing drives and rebalancing the array. But for the moment only the RAID 1 modes are really trustworthy. I have a NAS consisting of drives whose adverti…

The BTRFS raid1 feature is very badly named, its best called two-copy. Skipping some details, two copies of data are written to the two drives with the most free space. So you can have multiple mismatched drives.

However you get no striping, and data is only read from one drive, so performance is limited to that of one drive for reads and writes. Plus with mismatched drives, smaller drives go unused unless you write enough data.

Re: OpenZFS – add disks to existing RAIDZ

#102

Post by nortonham »

jedberg wrote: post #30

I wish Apple and Oracle would have just sorted things out and made ZFS the main filesystem for the Mac. Way back in the day when they first designed Time Machine, it was supposed to just be a GUI for ZFS snapshots. How cool would it be if we had a great GUI for ZFS (snapshots, volume management, etc.). I could buy a new external disk, add it to a pool, have seamless storage expansion. It would be great. Ah, what coul…

>How cool would it be if we had a great GUI for ZFS (snapshots, volume management, etc.)

I believe that was something built into the file manager in OpenSolaris, and then into illumos OS's. OpenIndiana has it in their file manager within their themed MATE desktop environment.

Re: OpenZFS – add disks to existing RAIDZ

#103

Post by wtallis »

keeperofdakeys wrote: post #101
wtallis wrote: post #67

Earlier quoted context omitted.

> which rules out all RAID levels except 0, which is nuts. RAID 1, you mean? Because that way you still have a complete copy of your data if one drive fails. BTRFS is excellent for the use case of a wide variety of mismatched drives, because it supports adding and removing drives and rebalancing the array. But for the moment only the RAID 1 modes are really trustworthy. I have a NAS consisting of drives whose adverti…

The BTRFS raid1 feature is very badly named, its best called two-copy. Skipping some details, two copies of data are written to the two drives with the most free space. So you can have multiple mismatched drives. However you get no striping, and data is only read from one drive, so performance is limited to that of one drive for reads and writes. Plus with mismatched drives, smaller drives go unused unless you write…

> The BTRFS raid1 feature is very badly named,

You're going to have to support that claim a bit better. The core idea of RAID 1 is mirroring data, which BTRFS RAID 1 mode definitely does. Striping is not an essential part of RAID 1 (hence RAID 10), and reading data from two disks in parallel is an optional performance optimization that is not performed by all RAID 1 implementations (but could be implemented for BTRFS RAID 1: https://stackoverflow.com/questions/55408256/btrfs-raid-1-wh... ).

> Plus with mismatched drives, smaller drives go unused unless you write enough data.

Yes, the allocation is suboptimal from a performance perspective, as I've already said. But it is simple and straightforward and reasonably good at avoiding putting you into a situation where manually issuing a rebalance command is necessary. If you do need better performance, there's a RAID 10 mode. But since my NAS is currently a motley pile of SSDs, I don't need to to anything extra to have decent performance.

Re: OpenZFS – add disks to existing RAIDZ

#104

Post by RulerOf »

gregmac wrote: post #48
hinkley wrote: post #36

I’m frustrated because this feature was mentioned by Schwartz when it was still in beta. I thought a new era of home computing was about to start. It didn’t, and instead we got The Cloud, which feels like decentralization but is in fact massive centralization (organizational, rather than geographical). Some of us think people should be hosting stuff from home, accessible from their mobile devices. But the first and t…

I can afford it, but have a hard time justifying the costs, not to mention scrapped (working) hardware and inconvenience (of swapping to a whole new array). I started using snapraid [1] several years ago, after finding zfs couldn't expand. Often when I went to add space the "sweet spot" disk size (best $/TB) was 2-3x the size of the previous biggest disk I ran. This was very economical compared to replacing the whole…

I gave snapraid a serious look a few months back and decided it might not be for me because the act of balancing writes out to the "array" member disks appeared to be manual.

I didn't want to point applications at 100 T of "free" space only for attires to start blocking after 8.

Am I mistaken about that?

Re: OpenZFS – add disks to existing RAIDZ

#105

Post by rincebrain »

Terretta wrote: post #59
jedberg wrote: post #30

I wish Apple and Oracle would have just sorted things out and made ZFS the main filesystem for the Mac. Way back in the day when they first designed Time Machine, it was supposed to just be a GUI for ZFS snapshots. How cool would it be if we had a great GUI for ZFS (snapshots, volume management, etc.). I could buy a new external disk, add it to a pool, have seamless storage expansion. It would be great. Ah, what coul…

> How cool would it be if we had a great GUI for ZFS (snapshots, volume management, etc.). I could buy a new external disk, add it to a pool, have seamless storage expansion. See QNAP HERO 5: https://www.qnap.com/static/landing/2021/quts-hero-5.0/en/in... NAS Options: https://www.qnap.com/en-us/product/?conditions=4-3 // This was about MacOS, and yes, that would be cool. But QNAP is remarkably MacOS-friendly, includi…

Worth noting QNAP's pools are 500% not ever going to be importable on stock OpenZFS.

If you're happily using their devices, this may not matter to you, but since the post is about OpenZFS...

Re: OpenZFS – add disks to existing RAIDZ

#106

Post by rincebrain »

jonhohle wrote: post #55
rincebrain wrote: post #26

Earlier quoted context omitted.

I don't think Linux would like to ship a mass of code that size that's not GPLed, even if court cases say CDDL is GPL-compatible.

It isn’t compatible because it adds additional restrictions regarding patented code. The CDDL was made to be used in mixed license distributions and only affects individual files, but the GPL taints anything linked (which is why LGPL exists). Since the terms of the CDDL can’t be respected in a GPL’d distribution, I can’t see a way for it to ever be included in the kernel repo. I don’t think there’s any issue with can…

I'm aware of the surrounding arguments, I promise; my remark was that I doubted Linux would like to ship a large mass of code with any additional encumberments even if it didn't somehow fail the GPL compatible test. (I'm aware that this is an impossible condition barring court precedent that ignores the definition of the constraint.)

Re: OpenZFS – add disks to existing RAIDZ

#107

Post by rincebrain »

katbyte wrote: post #69
crote wrote: post #29

> After the expansion completes, old blocks remain with their old data-to-parity ratio (e.g. 5-wide RAIDZ2, has 3 data to 2 parity), but distributed among the larger set of disks. New blocks will be written with the new data-to-parity ratio (e.g. a 5-wide RAIDZ2 which has been expanded once to 6-wide, has 4 data to 2 parity). Does anyone know why this is the case? When expanding an array which is getting full this wi…

I imagine there would eventually be a way / option to automatically rewrite old blocks. Or at least I would hope so because usually when your adding new disks the array is going to be near full

Almost certainly not in any useful fashion that doesn't duplicate data versus old snapshots.

ZFS really deeply assumes you're not gonna be rewriting history for a bunch of features, and you'd have written a good chunk of a new filesystem to reimplement everything without those assumptions.

Re: OpenZFS – add disks to existing RAIDZ

#108

Post by keeperofdakeys »

wtallis wrote: post #103
keeperofdakeys wrote: post #101

Earlier quoted context omitted.

The BTRFS raid1 feature is very badly named, its best called two-copy. Skipping some details, two copies of data are written to the two drives with the most free space. So you can have multiple mismatched drives. However you get no striping, and data is only read from one drive, so performance is limited to that of one drive for reads and writes. Plus with mismatched drives, smaller drives go unused unless you write…

> The BTRFS raid1 feature is very badly named, You're going to have to support that claim a bit better. The core idea of RAID 1 is mirroring data, which BTRFS RAID 1 mode definitely does. Striping is not an essential part of RAID 1 (hence RAID 10), and reading data from two disks in parallel is an optional performance optimization that is not performed by all RAID 1 implementations (but could be implemented for BTRFS…

RAID1 is about mirroring disks, BTRFS RAID1 mirrors block groups. Plus traditionally a RAID1 of 3 disks will mirror the same data on all disks, which is different to how the RAID1 mode on BTRFS acts. So the name leads to misunderstandings since it doesn't act like RAID1 at all.

It'd be way easier to talk about if it had a unique name, and you could say "It's like RAID1".

Despite all that I do like the mode, and use it in a few places.

Re: OpenZFS – add disks to existing RAIDZ

#109

Post by michaelmrose »

postmodest wrote: post #90
jedberg wrote: post #30

I wish Apple and Oracle would have just sorted things out and made ZFS the main filesystem for the Mac. Way back in the day when they first designed Time Machine, it was supposed to just be a GUI for ZFS snapshots. How cool would it be if we had a great GUI for ZFS (snapshots, volume management, etc.). I could buy a new external disk, add it to a pool, have seamless storage expansion. It would be great. Ah, what coul…

Apple wanted one FS across all devices and ZFS's ARC makes that less than useful in memory-starved phones (and base model Airs amirite?) ZFS's separate cache meant that the better choice was APFS. Once they got around to writing it. And on drives at least, TM backups ARE APFS snapshots.

Basically every OS has a file cache and you can tune the ARC how you please

Re: OpenZFS – add disks to existing RAIDZ

#110

Post by gregmac »

RulerOf wrote: post #104
gregmac wrote: post #48

Earlier quoted context omitted.

I can afford it, but have a hard time justifying the costs, not to mention scrapped (working) hardware and inconvenience (of swapping to a whole new array). I started using snapraid [1] several years ago, after finding zfs couldn't expand. Often when I went to add space the "sweet spot" disk size (best $/TB) was 2-3x the size of the previous biggest disk I ran. This was very economical compared to replacing the whole…

I gave snapraid a serious look a few months back and decided it might not be for me because the act of balancing writes out to the "array" member disks appeared to be manual. I didn't want to point applications at 100 T of "free" space only for attires to start blocking after 8. Am I mistaken about that?

Sorry I said unionfs but it's actually handled by mergerfs [1], and it's all automatic. There are a whole boatload of policies [2] to control writes.

I use "existing path, least free space". Once a path is created, it keeps using it for new files in that path. If it runs out of space, it creates that same path on another drive. If the path exists on both drives for some reason, my rationale is this keeps most of the related files (same path) together on the same drive.

I see there's some newer "most shared path" options I don't remember that might even make more sense for me, so maybe that's something I'll change next time I need to touch it.

[1] https://github.com/trapexit/mergerfs

[2] https://github.com/trapexit/mergerfs#policy-descriptions

Post reply on HN