Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

ZFS won’t save you: fancy filesystem fanatics need to get a clue about bit rot

nctritech.wordpress.com

Post reply on HN
Print view View original

11–20 of 38 posts

Re: ZFS won’t save you: fancy filesystem fanatics need to get a clue about bit rot

#11

Post by Veratyr »

While he's right that it's not as big an issue as ZFS fanatics make it out to be, it _is_ a real issue and they're not just pulling it out their asses. There are a number of studies that actually measured the error rate, some of the bigger ones being done by CERN [0], NetApp [1] and IA (I think there's meant to be a talk or something to go with this one) [2].

ZFS certainly isn't a magic wand you should wave at anything and everything and it doesn't replace backups but it does make the chances of something going wrong undetected much smaller and even though the chances are small to begin with, there are times when you just can't accept it at all.

[0]: https://www.nsc.liu.se/lcsc2007/presentations/LCSC_2007-kele...

[1]: https://www.usenix.org/legacy/events/fast08/tech/full_papers...

[2]: http://storageconference.us/2006/Presentations/39rWFlagg.pdf

Re: ZFS won’t save you: fancy filesystem fanatics need to get a clue about bit rot

#12

Post by Veratyr »

random_comment wrote: post #9

This entire article can be summarised as 'guy who has never used ZFS and has no idea whatsoever about how it works writes a critique that exposes their ignorance publicly'. Here's a quote: - “ZFS has CRCs for data integrity A certain category of people are terrified of the techno-bogeyman named “bit rot.” These people think that a movie file not playing back or a picture getting mangled is caused by data on hard driv…

> No, they don't, or Oracle wouldn't have spent money making it.

Tiny nitpick but though Oracle now owns and develops ZFS, Sun Microsystems was the company that initially designed and implemented it. They worked on it for 5 years after they released it, before Oracle acquired them.

Re: ZFS won’t save you: fancy filesystem fanatics need to get a clue about bit rot

#14

Post by X86BSD »

Veratyr wrote: post #11

While he's right that it's not as big an issue as ZFS fanatics make it out to be, it _is_ a real issue and they're not just pulling it out their asses. There are a number of studies that actually measured the error rate, some of the bigger ones being done by CERN [0], NetApp [1] and IA (I think there's meant to be a talk or something to go with this one) [2]. ZFS certainly isn't a magic wand you should wave at anythi…

Actually it does replace backups with replication and/or cloning.

Re: ZFS won’t save you: fancy filesystem fanatics need to get a clue about bit rot

#15

Post by random_comment »

Veratyr wrote: post #12
random_comment wrote: post #9

This entire article can be summarised as 'guy who has never used ZFS and has no idea whatsoever about how it works writes a critique that exposes their ignorance publicly'. Here's a quote: - “ZFS has CRCs for data integrity A certain category of people are terrified of the techno-bogeyman named “bit rot.” These people think that a movie file not playing back or a picture getting mangled is caused by data on hard driv…

> No, they don't, or Oracle wouldn't have spent money making it. Tiny nitpick but though Oracle now owns and develops ZFS, Sun Microsystems was the company that initially designed and implemented it. They worked on it for 5 years after they released it, before Oracle acquired them.

Whoops, thanks for the catch. Have updated and also added OpenZFS to that sentence.

Re: ZFS won’t save you: fancy filesystem fanatics need to get a clue about bit rot

#16

Post by Buge »

kabdib wrote: post #3

> While it is true that keeping a hash of a chunk of data will tell you if that data is damaged or not, the filesystem CRCs are an unnecessary and redundant waste of space ... A few years ago I, when I was on a game console team, a hardware engineer came to my desk and said, "Can you find out what's wrong with this disk drive?" It had come from a customer whose complaint was that games sometimes failed to download an…

But did the console's software checking help in that case? Either way you're going to have a customer complaining about problems.

Re: ZFS won’t save you: fancy filesystem fanatics need to get a clue about bit rot

#17

Post by ATsch »

>Snapshots may help, but they depend on the damage being caught before the snapshot of the good data is removed. If you save something and come back six months later and find it’s damaged, your snapshots might just contain a few months with the damaged file and the good copy was lost a long time ago.

The author seems to misunderstand the purpose of snapshots. As frequently [1] pointed out, snapshots are not in fact backups and should not be used for longer term storage.

Also the same argument can be used on Backups: "Backups may help, but they depend on the damage being caught before the backup of the good data is removed. If you save something and come back six months later and find it’s damaged, your backups might just contain a few months with the damaged file and the good copy was lost a long time ago."

[1] http://www.cobaltiron.com/2014/01/06/blog-snapshots-are-not-...

Re: ZFS won’t save you: fancy filesystem fanatics need to get a clue about bit rot

#18

Post by AstralStorm »

rgbrenner wrote: post #10
kabdib wrote: post #3

> While it is true that keeping a hash of a chunk of data will tell you if that data is damaged or not, the filesystem CRCs are an unnecessary and redundant waste of space ... A few years ago I, when I was on a game console team, a hardware engineer came to my desk and said, "Can you find out what's wrong with this disk drive?" It had come from a customer whose complaint was that games sometimes failed to download an…

Just asserting "CRCs are useless" is putting a lot of trust on stuff that has real-world failure modes. Yes, and he does this over and over again throughout the article. I have personally experienced at least 3 scenarios that he has determined won't happen. If this guy wrote a filesystem (something that he pretends to have enough experience to critique), it would be an unreliable unusable piece of crap.

You have worse problems that a filesystem won't catch if RAM gets randomly corrupted. Including said CRC check itself getting corrupted or code writing putt data structures to disk being wrong. Neither of those is caught by CRC better than by a dirty bit. It so happens that journaling file systems already have a degree of redundancy for writes built into them unless you defeat it.

Re: ZFS won’t save you: fancy filesystem fanatics need to get a clue about bit rot

#19

Post by AstralStorm »

Mindless2112 wrote: post #7

As someone who has lost some files to a silently malfunctioning hard disk in the past, I think I'll stick with ZFS. Checksumming, RAID-Z, and periodic scrubbing would have saved my files. Even having backups did not -- after all, what good is a bit-for-bit copy of a corrupted file? (On a side note, ZFS -- at least OpenZFS -- doesn't support any CRC algorithms for use as its checksum.)

Mostly periodic scrubbing and patrol reads I reckon. Which is as required with RAID without ZFS.

Re: ZFS won’t save you: fancy filesystem fanatics need to get a clue about bit rot

#20

Post by AstralStorm »

rgbrenner wrote: post #6

For an article with that tone, you would think the author would have more experience. It's literally filled with flawed and uninformed or inexperienced thinking. From the idea that SMART reliably detects hard drive failures.. to dismissing data protection for no reason other than it sounds unlikely to the author (which in several cases I know personally to be false... because I've experienced those failures). ZFS is…

Almost all of the protections are also afforded by plain old RAID without ZFS. Why waste space on a CRC when you still get to run a redundancy check? If FS structure is corrupted CRC won't save you anyway. An FSCK might instead.
Post reply on HN