Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

ZFS 2.3 released with ZFS raidz expansion

github.com

Post reply on HN
Print view View original

101–110 of 331 posts

Re: ZFS 2.3 released with ZFS raidz expansion

#101

Post by ryao »

koverstreet wrote: post #94
mustache_kimono wrote: post #70

Earlier quoted context omitted.

> Technically speaking, bcachefs has been merged into the Linux Kernel - that makes your initial assertion wrong. bcachefs doesn't implement its erasure coding/RAID yet? Doesn't implement send/receive. Doesn't implement scrub/fsck. See: https://bcachefs.org/Roadmap , https://bcachefs.org/Wishlist/ btrfs is still more of a legit competitor to ZFS these days and it isn't close to touching ZFS where it matters. If the p…

Erasure coding is almost done; all that's missing is some of the device evacuate and reconstruct paths, and people have been testing it and giving positive feedback (especially w.r.t. performance). It most definitely does have fsck and has since the beginning, and it's a much more robust and dependable fsck than btrfs's. Scrub isn't quite done - I actually was going to have it ready for this upcoming merge window exc…

Hearing that it is missing some code for reconstruction makes it sound like it is missing something fairly important. The original purpose of parity RAID is to support reconstruction.

Re: ZFS 2.3 released with ZFS raidz expansion

#102

Post by MauritsVB »

poisonborz wrote: post #59

I just don't get it how the Windows world - by far the largest PC platform per userbase - still doesn't have any answer to ZFS. Microsoft had WinFS and then ReFS but it's on the backburner and while there is active development (Win11 ships some bits time to time) release is nowhere in sight. There are some lone warriors trying the giant task of creating a ZFS compatibility layer with some projects, but they are far f…

There is occasional talk of moving the Windows implementation of OpenZFS (https://github.com/openzfsonwindows/openzfs/releases) into an officially supported tier, though that will probably come after the MacOS version (https://github.com/openzfsonosx) is officially supported.

Re: ZFS 2.3 released with ZFS raidz expansion

#103

Post by mogoh »

GuB-42 wrote: post #73
poisonborz wrote: post #59

I just don't get it how the Windows world - by far the largest PC platform per userbase - still doesn't have any answer to ZFS. Microsoft had WinFS and then ReFS but it's on the backburner and while there is active development (Win11 ships some bits time to time) release is nowhere in sight. There are some lone warriors trying the giant task of creating a ZFS compatibility layer with some projects, but they are far f…

To be honest, the situation with Linux is barely better. ZFS has license issues with Linux, preventing full integration, and Btrfs is 15 years in the making and still doesn't match ZFS in features and stability. Most Linux distros still use ext4 by default, which is 19 years old, but ext4 is little more than a series of extensions on top of ext2, which is the same age as NTFS. In all fairness, there are few OS compon…

ZFS might be better then any other FS on Linux (I don't judge that).

But you must admit that the situation on Linux is quite better then on Windows. Linux has so many FS in main branch. There is a lot of development. BTRFS had a rocky start, but it got better.

Re: ZFS 2.3 released with ZFS raidz expansion

#104

Post by philjohn »

abrookewood wrote: post #92

Can someone provide details on this bit please? "Direct IO: Allows bypassing the ARC for reads/writes, improving performance in scenarios like NVMe devices where caching may hinder efficiency". ARC is based in RAM, so how could it reduce performance when used with NVMe devices? They are fast, but they aren't RAM-fast ...

Yes - interested in this too. Is this for both ARC and L2ARC, or just L2ARC?

Re: ZFS 2.3 released with ZFS raidz expansion

#105

Post by CoolCold »

ryao wrote: post #83
wil421 wrote: post #77

Earlier quoted context omitted.

How many of us are using single disks on our laptops? I have a NAS and use all of the above but that doesn’t help people with single drive systems. Or help me understand why I would want it on my laptop.

My thinkpad from college uses ZFS as its rootfs. The benefits are: * If the hard drive / SSD corrupted blocks, the corruption would be identified. * Ditto blocks allow for self healing. Usually, this only applies to metadata, but if you set copies=2, you can get this on data too. It is a poor man’s RAID. * ARC made the desktop environment very responsive since unlike the LRU cache, ARC resists cold cache effects from…

NTFS had compression since mot even sure when.

For other stuff, let that nerdy CorpIT handle your system.

Re: ZFS 2.3 released with ZFS raidz expansion

#106

Post by BSDobelix »

jeroenhd wrote: post #84
poisonborz wrote: post #75

Earlier quoted context omitted.

Way better data security, resilience against file rotting. This goes for both HDDs or SSDs. Copy-on-write, snapshots, end to end integrity. Also easier to extend the storage for safety/drive failure (and SSDs corrupt in a more sneaky way) with pools.

The data security and rot resilience only goes for systems with ECC memory. Correct data with a faulty checksum will be treated the same as incorrect data with a correct checksum. Windows has its own extended filesystem through Storage Spaces, with many ZFS features added as lesser used Storage Spaces options, especially when combined with ReFS.

On zfs there is the ARC (adaptive read cache), on non-zfs systems this "read cache" is called buffer, both reside in memory, so ECC is equally important for both systems.

Rot means changing bits without accessing those bits, and that's ~not possible with zfs, additionally you can enable check-summing IN the ARC (disabled by default), and with that you can say that ECC and "enterprise" quality hardware is even more important for non-ZFS systems.

>Correct data with a faulty checksum will be treated the same as incorrect data with a correct checksum.

There is no such thing as "correct" data, only a block with a correct checksum, if the checksum is not correct, the block is not ok.

Re: ZFS 2.3 released with ZFS raidz expansion

#107

Post by _factor »

jeroenhd wrote: post #84
poisonborz wrote: post #75

Earlier quoted context omitted.

Way better data security, resilience against file rotting. This goes for both HDDs or SSDs. Copy-on-write, snapshots, end to end integrity. Also easier to extend the storage for safety/drive failure (and SSDs corrupt in a more sneaky way) with pools.

The data security and rot resilience only goes for systems with ECC memory. Correct data with a faulty checksum will be treated the same as incorrect data with a correct checksum. Windows has its own extended filesystem through Storage Spaces, with many ZFS features added as lesser used Storage Spaces options, especially when combined with ReFS.

This has nothing to do with ZFS as a filesystem. It has integrity verification on duplicated raid configurations. If the system memory flips a bit, it will get written to disk like all filesystems. If a bit flips on a disk, however, it can be detected and repaired. Without ECC, your source of truth can corrupt, but this true of any system.

Re: ZFS 2.3 released with ZFS raidz expansion

#108

Post by BSDobelix »

xattt wrote: post #79
GuB-42 wrote: post #73

Earlier quoted context omitted.

To be honest, the situation with Linux is barely better. ZFS has license issues with Linux, preventing full integration, and Btrfs is 15 years in the making and still doesn't match ZFS in features and stability. Most Linux distros still use ext4 by default, which is 19 years old, but ext4 is little more than a series of extensions on top of ext2, which is the same age as NTFS. In all fairness, there are few OS compon…

ZFS on OS X was killed because of Oracle licensing drama. I don’t expect anything better on Windows either.

>ZFS on OS X was killed because of Oracle licensing drama.

Naa it was Jobs ego not the license:

>>Only one person at Steve Jobs' company announces new products: Steve Jobs.

https://arstechnica.com/gadgets/2016/06/zfs-the-other-new-ap...

Re: ZFS 2.3 released with ZFS raidz expansion

#109

Post by craftkiller »

dizhn wrote: post #78
nesarkvechnep wrote: post #35

Earlier quoted context omitted.

I'm trying to find a reason not to use ZFS at home.

Requirement for enterprise quality disks, huge RAM (1 gig per TB), ECC, at least x5 disks of redundancy. None of these are things, but people will try to educate you anyway. So use it but keep it to yourself. :)

No need to keep it to yourself. As you've mentioned, all of these requirements are misinformation so you can ignore people who repeat them (or even better, tell them to stop spreading misinformation).

For those not in the know:

You don't need to use enterprise quality disks. There is nothing in the ZFS design that requires enterprise quality disks any more than any other file system. In fact, ZFS has saved my data through multiple consumer-grade HDD failures over the years thanks to raidz.

The 1 gig per TB figure is ONLY for when using the ZFS dedup feature, which the ZFS dedup feature is widely regarded as a bad idea except in VERY specific use cases. 99.9% of ZFS users should not and will not use dedup and therefore they do not need ridiculous piles of ram.

There is nothing in the design of ZFS any more dangerous to run without ECC than any other filesystem. ECC is a good idea regardless of filesystem but its certainly not a requirement.

And you don't need x5 disks of redundancy. It runs great and has benefits even on single-disk systems like laptops. Naturally, having parity drives is better in case a drive fails but on single disk systems you still benefit from the checksumming, snapshotting, boot environments, transparent compression, incremental zfs send/recv, and cross-platform native encryption.

Re: ZFS 2.3 released with ZFS raidz expansion

#110

Post by stephen_g »

GuB-42 wrote: post #73
poisonborz wrote: post #59

I just don't get it how the Windows world - by far the largest PC platform per userbase - still doesn't have any answer to ZFS. Microsoft had WinFS and then ReFS but it's on the backburner and while there is active development (Win11 ships some bits time to time) release is nowhere in sight. There are some lone warriors trying the giant task of creating a ZFS compatibility layer with some projects, but they are far f…

To be honest, the situation with Linux is barely better. ZFS has license issues with Linux, preventing full integration, and Btrfs is 15 years in the making and still doesn't match ZFS in features and stability. Most Linux distros still use ext4 by default, which is 19 years old, but ext4 is little more than a series of extensions on top of ext2, which is the same age as NTFS. In all fairness, there are few OS compon…

I’m interested to know what ‘full integration’ does look like, I use ZFS in Proxmox (Debian-based) and it’s really great and super solid, but I haven’t used ZFS in more vanilla Linux distros. Does Proxmox have things that regular Linux is missing out on, or are there shortcomings and things I just don’t realise about Proxmox?
Post reply on HN