Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

ZFS 2.3 released with ZFS raidz expansion

github.com

Post reply on HN
Print view View original

211–220 of 331 posts

Re: ZFS 2.3 released with ZFS raidz expansion

#211

Post by bbatha »

craftkiller wrote: post #109
dizhn wrote: post #78

Earlier quoted context omitted.

Requirement for enterprise quality disks, huge RAM (1 gig per TB), ECC, at least x5 disks of redundancy. None of these are things, but people will try to educate you anyway. So use it but keep it to yourself. :)

No need to keep it to yourself. As you've mentioned, all of these requirements are misinformation so you can ignore people who repeat them (or even better, tell them to stop spreading misinformation). For those not in the know: You don't need to use enterprise quality disks. There is nothing in the ZFS design that requires enterprise quality disks any more than any other file system. In fact, ZFS has saved my data th…

> The 1 gig per TB figure is ONLY for when using the ZFS dedup feature, which the ZFS dedup feature is widely regarded as a bad idea except in VERY specific use cases. 99.9% of ZFS users should not and will not use dedup and therefore they do not need ridiculous piles of ram.

You also really don't need a 1GB for RAM unless you have a very high write volume. YMMV but my experience is that its closer to 1GB for 10TB.

Re: ZFS 2.3 released with ZFS raidz expansion

#212

Post by nickdothutton »

poisonborz wrote: post #59

I just don't get it how the Windows world - by far the largest PC platform per userbase - still doesn't have any answer to ZFS. Microsoft had WinFS and then ReFS but it's on the backburner and while there is active development (Win11 ships some bits time to time) release is nowhere in sight. There are some lone warriors trying the giant task of creating a ZFS compatibility layer with some projects, but they are far f…

OS development pretty much stopped around 2000. ZFS is from 2001. I don't count a new way to organise my photos or integrate with a search engine as "OS" though.

Re: ZFS 2.3 released with ZFS raidz expansion

#213

Post by wkat4242 »

zamadatix wrote: post #193
wkat4242 wrote: post #189

Earlier quoted context omitted.

The benefits of ZFS don't need multiple drives to be useful. I'm running ZFS on root for years now and snapshots have saved my bacon several times. Also with block checksums you can at least detect bitrot. And COW is always useful.

Windows manages volume snapshots on NTFS through VSS. I think ZFS snapshots are a bit "cleaner" of a design, and the tooling is a bit friendlier IMO, but the functionality to snapshot, rollback, and save your bacon is there regardless. Outside of the automatically enabled "System Restore" (which only uses VSS to snapshot specific system files during updates) I don't think anyone bothers to use it though. CoW, advance…

Yes I know that NTFS has snapshots, I mentioned that in another comment. I don't think NTFS is as relevant in comparison though. People who choose windows will have no interest in ZFS and vice versa (someone considering ZFS will not pick Windows).

And I don't think anyone bothers to use it due to the lack of user-facing tooling around it. If it would be as easy to create snapshots as it is on ZFS, more people would use it, I'm sure. It's just so amazing to try something out, screw up my system and just revert :P But VSS is more of a system API than a user-facing geature.

VSS is also used by backup software to quiet the filesystem by the way.

But yeah the others are great features. My main point was though that almost all the features of ZFS are very beneficial even on a single drive. You don't need an array to take advantage of Snapshots, the crash reliability that CoW offers, and checksumming (though you will lack the repair option obviously)

Re: ZFS 2.3 released with ZFS raidz expansion

#214

Post by wkat4242 »

unsnap_biceps wrote: post #210
wkat4242 wrote: post #181

Earlier quoted context omitted.

ZFS also had expansion for a long time but it was offline expansion. I don't know if btrfs has also had online for a long time? And shrinking no, that is a big missing feature in ZFS IMO. Understandable considering its heritage (large scale datacenters) but nevertheless an issue for home use. But raidz is rock-solid. Btrfs' raid is not.

Raidz wasn't able to be expanded in place before this. You were able to add to a pool that included a raidz vdev, but that raidz vdev was immutable.

Oh ok, I've never done this, but I thought it was already there. Maybe this was the original ZFS from Sun? But maybe I just remember it incorrectly, sorry.

I've used it on multi-drive arrays but I never had the need for expansion.

Re: ZFS 2.3 released with ZFS raidz expansion

#215

Post by chungy »

ryao wrote: post #179
rekoil wrote: post #152

Earlier quoted context omitted.

Yaeh it's a pretty huge caveat to be honest. Da1 Db1 Dc1 Pa1 Pb1 Da2 Db2 Dc2 Pa2 Pb2 Da3 Db3 Dc3 Pa3 Pb3 ___ ___ ___ Pa4 Pb4 ___ represents free space. After expansion by one disk you would logically expect something like: Da1 Db1 Dc1 Da2 Pa1 Pb1 Db2 Dc2 Da3 Db3 Pa2 Pb2 Dc3 ___ ___ ___ Pa3 Pb3 ___ ___ ___ ___ Pa4 Pb4 But as I understand it it would actually expand to: Da1 Db1 Dc1 Dd1 Pa1 Pb1 Da2 Db2 Dc2 Dd2 Pa2 Pb2 D…

ZFS RAID-Z does not have parity disks. The parity and data is interleaved to allow data reads to be done from all disks rather than just the data disks. The slides here explain how it works: https://openzfs.org/w/images/5/5e/RAIDZ_Expansion_2023.pdf Anyway, you are not entirely wrong. The old data will have the old parity:data ratio while new data will have the new parity:data ratio. As old data is freed from the vde…

To be fair, RAID5/6 don't have parity disks either. RAID2, RAID3, and RAID4 do, but they're all effectively dead technology for good reason.

I think it's easy for a lot of people to conceptualize RAID5/6 and RAID-Zn as having "data disks" and "parity disks" to wrap around the complicated topic of how it works, but all of them truly interleave and compute parity data across all disks, allowing any single disk to die.

I've been of two minds on the persistent myth of "parity disks" but I usually ignore it, because it's a convenient lie to understand your data is safe, at least. It's also a little bit the same way that raidz1 and raidz2 are sometimes talked about as "RAID5" and "RAID6"; the effective benefits are the same, but the implementation is totally different.

Re: ZFS 2.3 released with ZFS raidz expansion

#216

Post by pdimitar »

koverstreet wrote: post #201
pdimitar wrote: post #142

Earlier quoted context omitted.

FYI: the main reason I gave up on bcachefs is that I can't use devices with native 16K blocks. Hope that's coming this year. I have a bunch of old HDDs and SSDs and I could very easily assemble a spare storage server with about 4TB capacity. Already tested bcachefs with most of the drives and it performed very well. Also lack of ability to reconstruct seems like another worrying omission.

I wasn't aware there were actual users needing bs > ps yet. Cool :) That should be a completely trivial for bcachefs to support, it'll mostly just be a matter of finding or writing the tests.

Seriously? But... NVMe drives! I stopped testing because I only have one spare NVMe and couldn't use it with bcachefs.

If you or others can get it done I'm absolutely starting to use bcachefs the month after. I do need fast storage servers in my home office.

Re: ZFS 2.3 released with ZFS raidz expansion

#217

Post by elseless »

mrb wrote: post #140
jeroenhd wrote: post #84

Earlier quoted context omitted.

The data security and rot resilience only goes for systems with ECC memory. Correct data with a faulty checksum will be treated the same as incorrect data with a correct checksum. Windows has its own extended filesystem through Storage Spaces, with many ZFS features added as lesser used Storage Spaces options, especially when combined with ReFS.

"data security and rot resilience only goes for systems with ECC memory." No. Bad HDDs/SSDs or bad SATA cables/ports cause a lot more data corruption than bad RAM. And ZFS will correct these cases even without ECC memory. It's a myth that the data healing properties of ZFS are useless without ECC memory.

Precisely this. And don’t forget about bugs in virtualization layers/drivers — ZFS can very often save your data in those cases, too.

Re: ZFS 2.3 released with ZFS raidz expansion

#218

Post by EvanAnderson »

wkat4242 wrote: post #213
zamadatix wrote: post #193

Earlier quoted context omitted.

Windows manages volume snapshots on NTFS through VSS. I think ZFS snapshots are a bit "cleaner" of a design, and the tooling is a bit friendlier IMO, but the functionality to snapshot, rollback, and save your bacon is there regardless. Outside of the automatically enabled "System Restore" (which only uses VSS to snapshot specific system files during updates) I don't think anyone bothers to use it though. CoW, advance…

Yes I know that NTFS has snapshots, I mentioned that in another comment. I don't think NTFS is as relevant in comparison though. People who choose windows will have no interest in ZFS and vice versa (someone considering ZFS will not pick Windows). And I don't think anyone bothers to use it due to the lack of user-facing tooling around it. If it would be as easy to create snapshots as it is on ZFS, more people would u…

> I don't think NTFS is as relevant in comparison though. People who choose windows will have no interest in ZFS and vice versa (someone considering ZFS will not pick Windows).

ZFS on Windows, as a first-class supported-by-Microsoft option would be killer. It won't ever happen, but it would be great. (NTFS / VSS with filesystem/snapshot send/receive would "scratch" a lot of that "itch", too.)

> And I don't think anyone bothers to use it due to the lack of user-facing tooling around it. If it would be as easy to create snapshots as it is on ZFS, more people would use it, I'm sure. It's just so amazing to try something out, screw up my system and just revert :P But VSS is more of a system API than a user-facing geature.

VSS on NTFS is handy and useful but in my experience brittle compared to ZFS snapshots. Sometimes VSS just doesn't work. I've had repeated cases over the years where accessing a snapshot failed (with traditional unhelpful Microsoft error messages) until the host machine was rebooted. Losing VSS snapshots on a volume is much easier than trashing a ZFS volume.

VSS straddles the filesystem and application layers in a way that ZFS doesn't. I think that contributes to some of the jank (VSS writers becoming "unstable", for example). It also straddles hardware interfaces in a novel way that ZFS doesn't (using hardware snapshot functionality-- somewhat like using a GPU versus "software rendering"). I think that also opens up a lot of opportunity for jank, as compared to ZFS treating storage as dumb blocks.

Re: ZFS 2.3 released with ZFS raidz expansion

#219

Post by EvanAnderson »

mrighele wrote: post #56
endorphine wrote: post #25

Can someone describe why they would use ZFS (or similar) for home usage?

Good reasons for me: Checksums: this is even more important in home usage as the hardware is usually of lower quality. Faulty controllers, crappy cables, hard disks stored in a higher than advised temperature... many reasons for bogus data to be saved, and zfs handles that well and automatically (if you have redundancy) Snapshots: very useful to make backups and quickly go back to an older version of a file when mist…

> Snapshots: very useful to make backups and quickly go back to an older version of a file when mistakes are made

Piling on here: Sending snapshots to remote machines (or removable drives) is very easy. That makes snapshots viable as a backup mechanism (because they can exist off-site and offline).

Re: ZFS 2.3 released with ZFS raidz expansion

#220

Post by taskforcegemini »

ryao wrote: post #162
crest wrote: post #141

Earlier quoted context omitted.

One problem with your setup is that ZFS by design can't use a traditional *nix filesystem buffer cache. Instead it has to use its own ARC (adaptive replacement cache) with end-to-end checksumming, transparent compression, and copy-on-write semantics. This can lead to annoying performance problems when the two types of file system caches contest for available memory. There is a back pressure mechanism, but it effectiv…

Traditionally, you have the page cache on top of the FS and the buffer cache below the FS, with the two being unified such that double caching is avoided in traditional UNIX filesystems. ZFS goes out of its way to avoid the buffer cache, although Linux does not give it the option to fully opt out of it since the block layer will buffer reads done by userland to disks underneath ZFS. That is why ZFS began to purge the…

what's the best way to deal with this then? disable filecache of linux? I've tried disabling/minimizing arc in the past to avoid the oom reaper, but the arc was stubborn and its RAM usage remained as is
Post reply on HN