Worth noting that TrueNAS already supports this[0] (I assuming using 2.3.0rc3?). Not sure about the stability, but very exciting.
ZFS 2.3 released with ZFS raidz expansion
41–50 of 331 posts
Re: ZFS 2.3 released with ZFS raidz expansion
#42FINALLY! You can do borderline insane single-vdev setups like RAID-Z3 with 4 disks (3 Disks worth of redundancy) of the most expensive and highest density hard drives money can buy right now, for an initial effective space usage of 25% and then keep buying and expanding Disk by Disk, with the space demand growing, up to something like 12ish disks. Disk prices dropping as time goes on and a spread out failure chance w…
Yes but see my sibling comment. When you expand your array, your existing data will not be stored any more efficiently. To get the new parity/data ratios, you would have to force copies of the data and delete the old, inefficient versions, e.g. with something like this [1] My personal take is that it's a much better idea to buy individual complete raid-z configurations and add new ones / replace old ones (disk by dis…
I wish something like this would be build into ZFS, so snapshots and current access would not be broken.
Re: ZFS 2.3 released with ZFS raidz expansion
#43Re: ZFS 2.3 released with ZFS raidz expansion
#44Earlier quoted context omitted.
Doing this with the on disk data in a merkle tree is much harder than doing it on more conventional forms of storage. By the way, what does MD do when there is corrupt data on disk that makes it impossible to know what the correct reconstruction is during a reshape operation? ZFS will know what file was damaged and proceed with the undamaged parts. ZFS might even be able to repair the damaged data from ditto blocks.…
Well, then they made a design choice in their RAID implementation that made fairly reasonable things hard. I don't know what md does if the parity doesn't match up, no. (I've never ever had that happen, in more than 25 years of pretty heavy md use on various disks.)
I am not sure if reshaping is a reasonable thing. It is not so reasonable in other fields. In architecture, if you build a bridge and then want more lanes, you usually build a new bridge, rather than reshape the bridge. The idea of reshaping a bridge while cars are using it would sound insane there, yet that is what people want from storage stacks.
Reshaping traditional storage stacks does not consider all of the ways things can go wrong. Handling all of them well is hard, if not impossible to do in traditional RAID. There is a long history of hardware analogs to MD RAID killing parity arrays when they encounter silent corruption that makes it impossible to know what is supposed to be stored there. There is also the case where things are corrupted such that there is a valid reconstruction, but the reconstruction produces something wrong silently.
Reshaping certainly is easier to do with MD RAID, but the feature has the trade off that edge cases are not handled well. For most people, I imagine that risk is fine until it bites them. Then it is not fine anymore. ZFS made an effort to handle all of the edge cases so that they do not bite people and doing that took time.
Re: ZFS 2.3 released with ZFS raidz expansion
#45Earlier quoted context omitted.
I'm not yet familiar with zfs and couldn't find it in the release note: Does expansion only works with disk of the same size? Or is adding are bigger/smaller disks possible or do all disk need to have the same size?
You need to buy the same exact drive with the same capacity and speed. Your raidz vdev be as small and as slow as your smallest and slowest drive. btrfs and the new bcachefs can do RAID with mixed drives, but I can’t trust either of them with my data yet.
Just have backups. I used btrfs and zfs for different purposes. Never had any lost data or downtime with btrfs since 2016. I only use raid 0 and raid 1 and compression. Btrfs does not havr a hungry ram requirement.
Re: ZFS 2.3 released with ZFS raidz expansion
#46Earlier quoted context omitted.
Is this possible elsewhere (re: other filesystems)?
Btrfs
Except you shouldn’t use btrfs for any parity based raid if you value your data at all. In fact, I’m not aware if any vendor that has implemented btrfs with parity based raid, they all resort to btrfs on md.
Re: ZFS 2.3 released with ZFS raidz expansion
#47Can someone describe why they would use ZFS (or similar) for home usage?
I replicate my entire filesystem to a local NAS every 10 minutes using zrepl. This has already saved my bacon once when a WD_BLACK SN850 suddenly died on me [1]. It's also recovered code from some classic git blunders. It shouldn't be possible any more to lose data to user error or single device failure. We have the technology.
Re: ZFS 2.3 released with ZFS raidz expansion
#48Earlier quoted context omitted.
I'm not yet familiar with zfs and couldn't find it in the release note: Does expansion only works with disk of the same size? Or is adding are bigger/smaller disks possible or do all disk need to have the same size?
You need to buy the same exact drive with the same capacity and speed. Your raidz vdev be as small and as slow as your smallest and slowest drive. btrfs and the new bcachefs can do RAID with mixed drives, but I can’t trust either of them with my data yet.
You can run raid-z across partitions to utilize the full drive just like synology does with their “hybrid raid” - you just shouldn’t.
Re: ZFS 2.3 released with ZFS raidz expansion
#49Earlier quoted context omitted.
Yes but see my sibling comment. When you expand your array, your existing data will not be stored any more efficiently. To get the new parity/data ratios, you would have to force copies of the data and delete the old, inefficient versions, e.g. with something like this [1] My personal take is that it's a much better idea to buy individual complete raid-z configurations and add new ones / replace old ones (disk by dis…
I wish something like this would be build into ZFS, so snapshots and current access would not be broken.
True, but I have a gut feeling that a lot of these thorny issues would come up again:
Re: ZFS 2.3 released with ZFS raidz expansion
#50Earlier quoted context omitted.
I'm not yet familiar with zfs and couldn't find it in the release note: Does expansion only works with disk of the same size? Or is adding are bigger/smaller disks possible or do all disk need to have the same size?
As far as I understand, ZFS doesn't work at all with disks of differing sizes (in the same array). So if you try it, it just finds the size of the smallest disk, and uses that for all disks. So if you put an 8TB drive in an array with a bunch of 10TB drives, they'll all be treated as 8TB drives, and the extra 2TB will be ignored on those disks. However, if you replace the smallest disk with a new, larger drive, and r…
This is the case with any parity based raid, they just hide it or lie to you in various ways. If you have two 6TB dives and two 12TB drives in a single raid-6 array, it is physically impossible to have two drive parity once you exceed 12TB of written capacity. BTRFS and bcachefs can’t magically create more space where none exists on your 6TB drives. They resort to dropping to mirror protection for the excess capacity which you could also do manually with ZFS by giving it partitions instead of the whole drive.