Silent Data Corruption Is Real
changelog.complete.org
Silent Data Corruption Is Real
1–10 of 154 posts
Re: Silent Data Corruption Is Real
#2Just like data black markets.
Re: Silent Data Corruption Is Real
#3It really bugs me (and has for a while) that there is still no mainstream linux filesystem that supports data block checksumming. Silent corruption is not exactly new, and the odds of running into it have grown significantly as drives have gotten bigger. It's a bit maddening that nobody seems to care (or maybe I'm just looking in the wrong places)
(...sure, you could call zfs or btrfs "mainstream", I suppose, but when I say "mainstream" I mean something along the lines of "officially supported by RedHat". zfs isn't, and RH considers btrfs to still be "experimental".)
Re: Silent Data Corruption Is Real
#4interesting find! I wonder what would be a good safeguard to this. I feel like just backing up your data would offer something - but a file could silently change and become corrupted in the backup too.
Re: Silent Data Corruption Is Real
#5It really bugs me (and has for a while) that there is still no mainstream linux filesystem that supports data block checksumming. Silent corruption is not exactly new, and the odds of running into it have grown significantly as drives have gotten bigger. It's a bit maddening that nobody seems to care (or maybe I'm just looking in the wrong places) (...sure, you could call zfs or btrfs "mainstream", I suppose, but whe…
OpenSuSE (and SLES) has considered btrfs stable for a while.
Re: Silent Data Corruption Is Real
#6It really bugs me (and has for a while) that there is still no mainstream linux filesystem that supports data block checksumming. Silent corruption is not exactly new, and the odds of running into it have grown significantly as drives have gotten bigger. It's a bit maddening that nobody seems to care (or maybe I'm just looking in the wrong places) (...sure, you could call zfs or btrfs "mainstream", I suppose, but whe…
Btrfs has experienced some data loss bugs in recent memory. It looks like ZFS is the only remaining option.
https://www.phoronix.com/scan.php?page=news_item&px=Btrfs-Da...
https://www.spinics.net/lists/linux-btrfs/msg59190.html
https://www.mail-archive.com/linux-btrfs@vger.kernel.org/msg...
Re: Silent Data Corruption Is Real
#7interesting find! I wonder what would be a good safeguard to this. I feel like just backing up your data would offer something - but a file could silently change and become corrupted in the backup too.
Hm. You could make two backups and checksum each file and safe the checksum. Then you could compare regularly file contents with the initial checksum, if there is a mismatch copy from the other backup.
Re: Silent Data Corruption Is Real
#8Closed source firmware on drives contain bugs that corrupt data. Are there any drives, available anywhere, that have open source firmware?
Re: Silent Data Corruption Is Real
#9I'm not a database expert, but this seems like something I should worry about, at least a bit. Is this a problem if you store all your persistent data in a database like MySQL?
Re: Silent Data Corruption Is Real
#10Closed source firmware on drives contain bugs that corrupt data. Are there any drives, available anywhere, that have open source firmware?
I guess this is what ZFS is for. Don't trust the underlying storage.
But I am interested as well about your question.