Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Tell HN: ZFS silent data corruption bugfix – my research results

news.ycombinator.com

Post reply on HN
Print view View original

51–60 of 90 posts

Re: Tell HN: ZFS silent data corruption bugfix – my research results

#51

Post by moss2 »

frafra wrote: post #48
moss2 wrote: post #42

Earlier quoted context omitted.

I'm unreasonably annoyed you used etcetera instead of et cetera . 'Et' is Latin for 'and'. The term means "and other things", not "andotherthings". No, I have nothing important to add to the ZFS discussion.

etcetera is found in ~500 years old books, and et cetera itself comes from et caetera , from the Greek καὶ τὰ ἕτερα . Which one should we use then? Should we only use the Latin from 1 BCE/CE or the vulgar Latin that evolved after? :-) Maybe I do not feel it like a big issue, since many Latin languages use it a single word (such as Italian with eccetera , Spanish, Portuguese), due to the evolution of the language.

I guess even ~500 year old books can be wrong :-)

I'm just joking. This is just a pet peeve of mine. Thank you for the fun lesson in etymology.

Re: Tell HN: ZFS silent data corruption bugfix – my research results

#52

Post by gmokki »

sandreas wrote: post #11
antongribok wrote: post #4

Earlier quoted context omitted.

For me it was the previous data corruption bug [1] that killed any enthusiasm for ZoL. After that annoyances like the caching issues you mention and the constant kernel upgrades breaking DKMS on Fedora just stopped being worth it for me. I finally moved to btrfs earlier this year, and so far I'm glad I did. I run raid1 on my primary array, and raid5 on my off-site backup array at my mom's apartment connecting with Ta…

Does btrfs support native encryption?

It is coming: https://lwn.net/ml/linux-fsdevel/cover.1701468305.git.josef@...

The kernel fscrypt layer just needed a lot of refactoring first to support cow filesystems.

Re: Tell HN: ZFS silent data corruption bugfix – my research results

#53

Post by sirdvd »

godelski wrote: post #8

Is there any good beginner friendly documentation for zfs? I've started using it as a testing/learning NAS with a raspberry pi (cloning my google library to immich). It has not been a clearly easy process and errors are very much not clear. I recently extended a single drive to 2 and now I can't import due to corrupted metadata and reports bad disk but smartctl reports all fine. Stack overflow is all over the place a…

I find klarasystems articles and Mr Salter articles on arstechnica are a good start point

Re: Tell HN: ZFS silent data corruption bugfix – my research results

#54

Post by E39M5S62 »

michaelmrose wrote: post #29
antongribok wrote: post #23

Earlier quoted context omitted.

The packaging was a major reason for switching to btrfs. I run sudo dnf upgrade and that's it, my system is upgraded. Zero issues ever. With ZFS I had to pin to older kernel versions, not to mention a bunch of manual steps and cleanup after any major version upgrade (every 6-12 months). Re btrfs raid 5/6, yes everyone knows about this, and this is why I have it only on my backup system. My primary data which holds 15…

I wonder if a metapackage that always depends on kernel <= supported would resolve the issue by ensuring you don't need to pin a specific version manually.

That's the way Void Linux does it. That ensures that the default kernel series always works with ZFS and NVidia modules. If you want to go off-reservation, you can do so but you're on your own, then.

Re: Tell HN: ZFS silent data corruption bugfix – my research results

#55

Post by mwpmaybe »

KyleSanderson wrote: post #2

https://github.com/openzfs/zfs/pull/15529#pullrequestreview-... Honestly, ZFS is the best thing on the (Free)BSDs only... On Linux it doesn't even use the page cache, and you conflict severely with L2ARC. I know there's a variety of people who don't care, but still for real users it's not an actual option.

If my data is in ARC, why would I also want it to be in page cache?

L2ARC isn't typically recommended for many use-cases. Anyway, what's the conflict you're referencing?

Re: Tell HN: ZFS silent data corruption bugfix – my research results

#56

Post by Volundr »

antongribok wrote: post #25
nightfly wrote: post #21

Earlier quoted context omitted.

Zfs has checksumming and scrubs, which can catch lots of data corruption that (most?) other filesystems can't catch catch at all

I guess I meant it as having a separate repair tool, but yes, you're of course correct. To me it was always weird to not have a separate tool for being able to do an offline ZFS repair. With regards to data corruption, I mean, this is exactly why I moved to btrfs, because it's able to catch bit rot same as ZFS. I think ultimately this is why ZFS data corruption bugs always are such a big deal... It's because in a lot…

> To me it was always weird to not have a separate tool for being able to do an offline ZFS repair.

Have you ever asked yourself why those operations need to be done offline?

It's not that ZFS doesn't have these capabilities, it's that ZFS is doing these similar checks and repairs as part of its normal operation and more. If you can't mount a ZFS filesystem, odds are a similarly borked ext4 filesystem isn't going to be repaired by fsck.

I got bit by some pretty bad foot guns in BTRFS that has meant I steer pretty far away, but my understanding is these have been somewhat mitigated. I hope it continues to work well for you.

Re: Tell HN: ZFS silent data corruption bugfix – my research results

#57

Post by buildbot »

moss2 wrote: post #42
buildbot wrote: post #10

Earlier quoted context omitted.

I feel like there has been kind of a weird concerted effort to push that zfs is bad due to this bug and how trust has been lost etcetera etcetera - super annoying when most other filesystems just corrupt your data and nobody will ever know it happened. I’ve experienced bad data corruption on xfs, btrfs, ext2, and ext4. So far zfs is been nothing but perfect.

I'm unreasonably annoyed you used etcetera instead of et cetera . 'Et' is Latin for 'and'. The term means "and other things", not "andotherthings". No, I have nothing important to add to the ZFS discussion.

Did not know this, I don’t mind learning something. You’s think though that autocorrect would pick up on that…

Re: Tell HN: ZFS silent data corruption bugfix – my research results

#58

Post by kaba0 »

worthless-trash wrote: post #43
buildbot wrote: post #39

Earlier quoted context omitted.

But it's totally different than corruption in other filesystems. People are acting like upgrading to ZFS 2.2.0 ate all their data like XFS used to back in the bad old days. I remember once the power went out at my house, and the entire XFS filesystem was irreparably damaged. This bug is super hard to trigger, and has roots back to 2006 and only surfaced with a recent coreutils update. Yet there are numerous posts in…

> But it's totally different than corruption in other > filesystems. People are acting like upgrading to ZFS 2.2.0 > ate all their data like XFS used to back in the bad old > days. I remember once the power went out at my house, and > the entire XFS filesystem was irreparably damaged. Different bugs manifest differently due to the coding, this is just the same. ZFS wouldn't even have a hundredth of the testing of rea…

ZFS and ext4 is so completely different on a fundamental level that there is zero reason comparing them, it just shows your "expertise" on the topic..

Re: Tell HN: ZFS silent data corruption bugfix – my research results

#59

Post by kaba0 »

antongribok wrote: post #4
KyleSanderson wrote: post #2

https://github.com/openzfs/zfs/pull/15529#pullrequestreview-... Honestly, ZFS is the best thing on the (Free)BSDs only... On Linux it doesn't even use the page cache, and you conflict severely with L2ARC. I know there's a variety of people who don't care, but still for real users it's not an actual option.

For me it was the previous data corruption bug [1] that killed any enthusiasm for ZoL. After that annoyances like the caching issues you mention and the constant kernel upgrades breaking DKMS on Fedora just stopped being worth it for me. I finally moved to btrfs earlier this year, and so far I'm glad I did. I run raid1 on my primary array, and raid5 on my off-site backup array at my mom's apartment connecting with Ta…

NixOS has seamless ZFS integration -- it's the problem of the package manager/distro, not of the tool

Re: Tell HN: ZFS silent data corruption bugfix – my research results

#60

Post by yjftsjthsd-h »

E39M5S62 wrote: post #54
michaelmrose wrote: post #29

Earlier quoted context omitted.

I wonder if a metapackage that always depends on kernel <= supported would resolve the issue by ensuring you don't need to pin a specific version manually.

That's the way Void Linux does it. That ensures that the default kernel series always works with ZFS and NVidia modules. If you want to go off-reservation, you can do so but you're on your own, then.

Yep, same on NixOS; just set

    boot.kernelPackages = config.boot.zfs.package.latestCompatibleLinuxPackages;
And you're done.
Post reply on HN