I maintain database and object-storage clusters for a living. Dealing with failing storage is half my job.
> Especially when it’s the filesystem metadata which gets corrupted.
We're not talking about filesystem metadata, though. Filesystem metadata is all "of a piece" — if you have a checksumming filesystem, then you can't not checksum some of the filesystem metadata, because all the metadata lives in (the moral equivalent of) a single database file the filesystem maintains, and that database gets checksummed. It's all one data structure, where the checksumming is a thing you do to that data structure, not to individual nodes within it. (For a tree filesystem like btrfs, this would be the non-cryptographic equivalent of a merkle-tree hash.) The only way you could even potentially turn off filesystem features for some metadata (dirent, freelist, etc) nodes but not others, would be to split your filesystem into multiple filesystems.
No, to be clear, we're specifically talking about what happens inside the filesystem's extents. Those can experience corruption without that causing any undue issues, besides "the data you get from fread(3) is wrong." Unlike filesystem metadata, which is all required for the filesystem's integrity, a checksumming filesystem can choose whether to "look" inside file extents, or to treat them as opaque. And it can (in theory) make that choice per file, if it likes. From the FS's perspective, an extent is just a range of reserved disk blocks.
Now, an assumption: only storage arrays use spinning rust for anything any more. The only disk problems consumer devices face any more are SSD degradation problems, not HDD degradation problems.
(Even if you don't agree with this assumption by itself, it's much more clear-cut if you consider only devices operated by people willing to choose to use a filesystem that's not the default one for their OS.)
This assumption neatly cleaves the problem-space in two:
- How should a filesystem on a RAID array, set up for a business or prosumer use-case, deal with HDD faults?
- How should a single-device filesystem used in a consumer use-case deal with SDD faults?
The HDD-faults case comes down to: filesystem-level storage pool management with filesystem-driven redundant reads, with kernel blocking-read timeouts to avoid hangs, with async bad-sector remapping for timed out reads. Y'know: ZFS.
While the SDD-faults case comes down to: read the bad data. Deal with the bad data. You won't get any hangs, until the day the whole thing just stops working. The worst you'll get is bit-rot. And even then, it's rare, because NAND controllers use internal space for error-correction, entirely invisibly to the kernel. (See also: http://dtrace.org/blogs/ahl/2016/06/19/apfs-part5/)
In fact, in my own personal experience, the most likely cause of incorrect or corrupt data ending up on an SSD/NVMe disk, is that the CPU or memory of the system is bad, and so one or the other is corrupting the memory that will be written to disk before or during the write. (I've personally had this happen at least twice. What to look for to diagnose this: PCIe "link training" errors.)