Live data from Hacker News

My 71 TiB ZFS NAS After 10 Years and Zero Drive Failures

louwrentius.com

221–230 of 314 posts

Re: My 71 TiB ZFS NAS After 10 Years and Zero Drive Failures

#221

Earlier quoted context omitted.

It can be luck, but with 24 drives, it feels very lucky. Somebody with proper statistics knowledge can probably calculate the risk with a guestimated 1% yearly failure rate how likely it would be to have all 24 drives remaining. And remember, my previous NAS with 20 drives also didn't have any failures. So N=44, how lucky must I be? It's for residential usage, and if I need some data, I often just copy it over 10Gbit…

We don't really have to guess. Backblaze posted their stats for 4 TB HGST drives for 2024, and of their 10,000 drives, 5 failed. If OP's 2014 4 TB HGST drives are anything like this, then this is just snake oil and magic rituals and it doesn't really matter what you do.

Those are different drives though, they're MegaScale DC 4000 while OP is using 4 TB Deskstars. Not sure if they're basically the same (probably). I've also had a bunch of these 4TB Megascale drives and absolutely no problems whatsoever in about 10 years as well. Run very cool as well (I think they're 5400 rpm not 7200 rpm).

The main issue with drives like these is that 4 TB is just so little storage compared to 16-20 TB class drives, it kinda gets hard to justify the backplane slot.

Re: My 71 TiB ZFS NAS After 10 Years and Zero Drive Failures

#222
post #163

Earlier quoted context omitted.

That was a decision Linus regretted[1]. There has been some recent discussion about this here on Hacker News[2]. [1] https://linuxiac.com/torvalds-expresses-regret-over-merging-... [2] https://news.ycombinator.com/item?id=41407768

As usual, the top comments in that submission are very biased. I think HN should sort comments in a random order in every polarizing discussion. Anyone reading this, do yourself a favor and dig through both links, or ignore the parent's comment altogether. Linus "regretted" it in the sense "it was a bit too early because bcachefs is moving at such a fast speed", and not in the sense "we got a second btrfs that eats y…

Well. Point taken. You have an important core of truth to your argument about polarization.

But...

Strongly disagree.

I think that is a very unfair reading of what I wrote. I feel that you might have a bias which shows but that would be the same class of ad hominem as you have just displayed. That is why I choose to react even though it might be wise to let slepping dogs lie. We should minimize polarization but not to a degree where we cannot have civilized disagreement. You are then doing exactly what you preach not to do. Is that then FUD with FUD on top? Two wrongs make a right?

I was reacting on the implicit approval in mentioning that it had been upstreamed in the kernel. The reason for the first link. Regrets where clearly expressed.

Another HN trope is rehashing the same discussions over and over again. That was the reason for the second link. I would like to avoid yet another discussion on a topic which was put into light less than 14 days ago. Putting that more bluntly would have been impolite and polarizing. Yet here I am.

The sad part is that my point got through to you loud and clear. Sad because rather than simply dismissing as polarizing that would have been a great opener for a discussion. Especially in the context of ZFS and durability.

You wrote:

> Linus "regretted" it in the sense "it was a bit too early because bcachefs is moving at such a fast speed", and not in the sense "we got a second btrfs that eats your data for lunch".

If you allow me a little lighthearted response. The first thing which comes to mind was the "They're the same picture" meme[1] from The Office. Some like to move quickly and break things. That is a reasonable point of view. But context matters. For long term data storage I am much more conservative. So while you might disagree; to me it is the exact same picture.

Hence I very much object to what I feel is an ad hominem attack because your own worldview was not reflected suitably in my response. It is fair critique that you feel it is FUD. I do however find it warranted for a filesystem which is marked experimental. It might be the bees knees but in my mind it is not ready for mainstream use. Yet.

That is an important perspective for the OP to have. If the OP just want to play around all is good. If the OP does not mind moving quickly and break things, fine. But for production use? Not there yet. Not in my world.

Telling people to ignore my comment because you know people cannot be bothered to actually read the links? And then lecturing me that people might take the wrong spin on it? Please!

[1] https://knowyourmeme.com/memes/theyre-the-same-picture

Re: My 71 TiB ZFS NAS After 10 Years and Zero Drive Failures

#223

Earlier quoted context omitted.

> we should stick to filesystems with data checksums such as ZFS, as well as ECC memory. While I don't disagree with this statement, consider the reality: - APFS has metadata checksums, but no data checksums. WTF Apple? - Very few Linux distributions ship zfs.ko (&spl.ko); those that do, theoretically face a legal risk (any kernel contributor could sue them for breaching the GPL); rebuilding the driver from source is…

> - Very few Linux distributions ship zfs.ko (&spl.ko); those that do, theoretically face a legal risk (any kernel contributor could sue them for breaching the GPL) > - Linux itself explicitly treats ZFS as unsupported; loading the module taints the kernel. So modify the ZFS source so it appears as a external GPL module... just don't tell anyone or distribute it... I can't say much about dracut or having to build the…

> So modify the ZFS source

Way to miss the point

Re: My 71 TiB ZFS NAS After 10 Years and Zero Drive Failures

#224

Earlier quoted context omitted.

Except for the helium-filled ones, they aren't sealed; there is a very fine filter that equalises atmospheric pressure. (This is also why they have a maximum operating altitude --- the head needs a certain amount of atmospheric pressure to float.)

How does the helium stay in if it is not sealed? I am not familiar with hard drive construction, but helium is notoriously good at escaping.

I think he meant in general drives aren't sealed, except the helium ones are sealed.

Re: My 71 TiB ZFS NAS After 10 Years and Zero Drive Failures

#225

Earlier quoted context omitted.

> - Very few Linux distributions ship zfs.ko (&spl.ko); those that do, theoretically face a legal risk (any kernel contributor could sue them for breaching the GPL) > - Linux itself explicitly treats ZFS as unsupported; loading the module taints the kernel. So modify the ZFS source so it appears as a external GPL module... just don't tell anyone or distribute it... I can't say much about dracut or having to build the…

> So modify the ZFS source Way to miss the point

Not really... the complaint was over licensing and tainting the kernel... so just tell the kernel it's not a CDDL module... problem solved.

Re: My 71 TiB ZFS NAS After 10 Years and Zero Drive Failures

#227
post #171
post #163

Earlier quoted context omitted.

That was a decision Linus regretted[1]. There has been some recent discussion about this here on Hacker News[2]. [1] https://linuxiac.com/torvalds-expresses-regret-over-merging-... [2] https://news.ycombinator.com/item?id=41407768

Context. Linux regrets it because bcachefs doesn't have same commitment to stability as Linux. Kent wants to fix a bug with large PR Linux doesn't want to merge and review PR that touches so many non-bcachefs things. They're both right in a way. Kent wants bcachefs to be stable/work good, Linus wants Linux to be stable.

Edit: replied to wrong person. I agree with you.

Kent from bcachefs was just late in the cycle, somewhere in rc5. That was indeed too late for such a huge push of new code touching so many things.

There is some tension but there is no drama and implying so is annoying.

Bcachefs is going places, I think I’d already choose it over btrfs atm.

Re: My 71 TiB ZFS NAS After 10 Years and Zero Drive Failures

#228
post #214

Earlier quoted context omitted.

Reminds me of the time back in the day when Dell shipped us a server with drives serial numbers being consecutive. Of course both failed at the same time and I spent an all nighter doing a restore.

I ordered my NAS drives on Amazon, to avoid getting the same batch (all consecutive serial numbers) I used amazon.co.uk for one half and amazon.de for the other half of them. One could also stage the orders in time.

Yeah, the risk of the rest of the old drives failing under high load while rebuilding/restoring is also very real, so staging is necessary as well.

I don't exactly hoard data by dozens of terabytes, but I rotate my backup drives each few years, with a 2-year difference between them.

Re: My 71 TiB ZFS NAS After 10 Years and Zero Drive Failures

#229

Earlier quoted context omitted.

> we should stick to filesystems with data checksums such as ZFS, as well as ECC memory. While I don't disagree with this statement, consider the reality: - APFS has metadata checksums, but no data checksums. WTF Apple? - Very few Linux distributions ship zfs.ko (&spl.ko); those that do, theoretically face a legal risk (any kernel contributor could sue them for breaching the GPL); rebuilding the driver from source is…

> I never had the guts to try btrfs in production after all the horror stories I've heard over the decade+. I've been running btrfs as the primary filesystem for all of my desktop machines since shortly after the on-disk format stabilized and the extX->btrfs in-place converter appeared [0], and for my home servers for the past ~five years. In the first few years after I started using it on my desktop machines, I had…

>At $DAYJOB we use btrfs as the filesystem for our CI workers and have been doing so for years. Its snapshot functionality makes creating the containers for CI jobs instantaneous, and we've had zero problems with it.

$DAYJOB == "facebook"?

Re: My 71 TiB ZFS NAS After 10 Years and Zero Drive Failures

#230
post #226

Having 24 drives probably offers some performance advantages, but if you don‘t require them, having a 6-bay NAS with 18TB disks instead woukld offer a ton of advantages in terms of power usage, noise, space required, cost and reliability.

I agree, that’s what I would do today if I wanted the same amount of space, as I states in the article.
Post reply on HN