Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A data corruption bug in OpenZFS?

despairlabs.com

Post reply on HN
Print view View original

1–10 of 115 posts

Re: A data corruption bug in OpenZFS?

#3

Post by MenhirMike »

Periodic reminder to check if your backups are working, and if you can also restore them. It doesn't matter which file system or operating system you use, make sure to backup your stuff. In a way that's immune to ransomware as well, so not just a RAID-1/5/Z or another form of hot/warm storage (RAID is not a backup, it's an uptime/availability mechanism) but cold storage. (I snapshot and tar that snapshot every night, then back it up both on tape and in the cloud.)

Re: A data corruption bug in OpenZFS?

#4

Post by moviuro »

dannyw wrote: post #2

Fascinating write up. As someone with a ZFS system, how can I check if I’m affected?

It's a very rare race condition, odds are very low that you were impacted. If you were, you would have noticed (heavy builds with files being moved around where suddenly files are zero).

[0] https://bugs.gentoo.org/917224

[1] https://github.com/openzfs/zfs/issues/15526 (referenced in the article)

Re: A data corruption bug in OpenZFS?

#5

Post by bgro »

MenhirMike wrote: post #3

Periodic reminder to check if your backups are working, and if you can also restore them. It doesn't matter which file system or operating system you use, make sure to backup your stuff. In a way that's immune to ransomware as well, so not just a RAID-1/5/Z or another form of hot/warm storage (RAID is not a backup, it's an uptime/availability mechanism) but cold storage. (I snapshot and tar that snapshot every night,…

It’s always amazing to me how frequently backups silently fail. Every backup software or general common tool to back things up that I’ve seen has many points of silent failure where it just gives up copying at some point in the process or skips over files for some reason without indicating what or why.

If you don’t delete files as you go, now you have an unknown partial backup state that basically doubles your needed space.

If you delete as you go, sometimes something happens and the process stops or corrupts so your data is now split and you may have lost something.

Even trying to log all the failures during the process is amazingly difficult and solutions to work around that specific problem, themselves, somehow introduce more and new types silent failure in some type of irony.

Re: A data corruption bug in OpenZFS?

#7

Post by hulitu »

> This whole madness started because someone posted an attempt at a test case for a different issue, and then that test case started failing on versions of OpenZFS that didn’t even have the feature in question.

One will expect more seriosity from filesystem maintainers and serious regression testing before a release.

Post reply on HN