Probably obvious, but hooray for open source software! What a fantastic response to the bug.
Imagine if this had happened to Apple, everyone and his dog would be rolling over each other villifying them.
ZFS on Linux: Unlistable and disappearing files
91–100 of 168 posts
Re: ZFS on Linux: Unlistable and disappearing files
#92Earlier quoted context omitted.
And that you regularly test that property.
But how do you test that reliably if the filesystem makes files disappear? I.e., you'll have an original that misses some files, and a backup that misses the same files. A diff will show nothing.
Unlikely to be the same files
Re: ZFS on Linux: Unlistable and disappearing files
#93Earlier quoted context omitted.
And that you regularly test that property.
But how do you test that reliably if the filesystem makes files disappear? I.e., you'll have an original that misses some files, and a backup that misses the same files. A diff will show nothing.
Compare an older snapshot to current for unexpected changes? Though that is difficult (probably not practically possible in the general case) to automate because telling the difference between rare but expected changes and unexpected ones due to corruption could require quite some intelligence/intuition.
No backup+test stratergey can catch everything. You just have to be sure to catch everything that you practically can (and need to factor your time & effort and the importance of the data, into the judgement about that is practical to do and what is justifiable or unavoidable risk).
Re: ZFS on Linux: Unlistable and disappearing files
#94At least it's no btrfs. What a disaster that filesystem's been.
Agree. Good old NIH.
Re: ZFS on Linux: Unlistable and disappearing files
#95Why is the thread title hyphenated? It made me expect there's a utility called "zfs-bug" causing data loss.
The OP appears to speak German natively. In German they are much more hyphen happy than English. It's probably the most common error I see among German speakers typing English.
Too many Germans use "idiot spaces" ("Deppenleerzeichen") though, e.g. "Adler Straße" instead of the correct "Adlerstraße" or "Adler-Straße" ("Eagle Street"). Anglophony hasn't done us good.
Re: ZFS on Linux: Unlistable and disappearing files
#96Why is the thread title hyphenated? It made me expect there's a utility called "zfs-bug" causing data loss.
The OP appears to speak German natively. In German they are much more hyphen happy than English. It's probably the most common error I see among German speakers typing English.
Exactly this. In German the headline would just have been:
"Linuxzfsdatenverlustfehler gedfunden!"
Getting this stuff right in English is hard. (Pun intended)
Sry!
Re: ZFS on Linux: Unlistable and disappearing files
#97So this is not a "ZFS bug", but a "ZFS on Linux" bug. The actual ZFS on systems which have had ZFS for decades is not affected at all.
It is a recent regression from 2 months ago that originated in the Linux port. I do not know the status of the other platforms. I and the others involved with the Linux port are still busy handling the issue. I suggest asking the developers of the other platforms whether they had adopted the bad patch or not.
I noticed someone has fixed the title, so my comment has lost its point now. Awesome.
Re: ZFS on Linux: Unlistable and disappearing files
#98Earlier quoted context omitted.
> We will likely have some way to recover the orphaned files (like ext4’s lost+found) and fix the directory sizes in the very near future. How should people behave right now? Will normal usage of production filesystems erase data, or will read/write activity leave the potentially-orphaned files in place? You've also mentioned snapshots being tricky in the thread. Should people stop creating snapshots in case orphaned…
> 8.23+ randomized the order (I don't quite understand why) It's using inode order, which speeds up things significantly on some filesystems: https://lists.gnu.org/archive/html/bug-gnulib/2014-02/msg000...
Ah, thanks! I knew there was a rationale but hadn't quite gotten it.
Nice.
Re: ZFS on Linux: Unlistable and disappearing files
#99Earlier quoted context omitted.
But how do you test that reliably if the filesystem makes files disappear? I.e., you'll have an original that misses some files, and a backup that misses the same files. A diff will show nothing.
Unlikely to be the same files
In the general case, but in this case isn't unlikely at all: you have a ZFS pool that suffers from this bug, you back it up, the missing files doesn't show up neither in the original or in the backup.
Re: ZFS on Linux: Unlistable and disappearing files
#100At least it's no btrfs. What a disaster that filesystem's been.
i have been using btrfs in production for years now and it has never failed me, and i am doing hundreds of snapshots and send/receiving, i've reconfigured raids on the fly and went from 6 disk raid 10 of mixed size to raid 10 of same size. we have in some cases had many power failures with no data loss. We also have some set up using md raid and some using hardware raid... when i hear people dogging btrfs it just spe…
I use OpenSuse Tumbleweed with BTRFS on a laptop. This is nothing special - a 512GB SSD, no RAID, about 350GB of data and a bunch of snapshots. It was basically the out of the box configuration plus some snapper config to regularly snapshot /home.
One day I shut it down and it wouldn’t boot - the BTRFS filesystem had gotten itself into a state where it would mount ok read-only but hang the system when mounting read/write. My best guess is that I shut it down (meaning a normal shutdown through the GUI) while it was running its weekly rebalance in the background. I’ve since recovered the data and rebuilt the machine, but it’s the first time in a long time that I’ve had a filesystem (any filesystem) fail me.