Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

My 71 TiB ZFS NAS After 10 Years and Zero Drive Failures

louwrentius.com

Post reply on HN
Print view View original

71–80 of 314 posts

Re: My 71 TiB ZFS NAS After 10 Years and Zero Drive Failures

#71

Post by vunderba »

ed_mercer wrote: post #64

> Losing the system due to power shenanigans is a risk I accept. A UPS provides more than just that, it delivers constant energy without fluctuations and thus makes your hardware last longer.

Yeah, this definitely caused me to raise an eyebrow. UPS covers brown outs and obviously the occasional temporary power outage. All those drives spinning at full speed suddenly coming to a grinding halt as the power is suddenly cut, and you're quibbling over a paltry additional 10 watts? I can only assume that the data is not that important.

Re: My 71 TiB ZFS NAS After 10 Years and Zero Drive Failures

#72

Post by adastra22 »

foobarian wrote: post #69
turnsout wrote: post #8

I’ve heard the exact opposite advice (keep the drives running to reduce wear from power cycling). Not sure what to believe, but I like having my ZFS NAS running so it can regularly run scrubs and check the data. FWIW, I’ve run my 4 drive system for 10 years with 2 drive failures in that time, but they were not enterprise grade drives (WD Green).

> regularly run scrubs and check the data Does this include some kind of built-in hash/checksum system to record e.g. md5 sums of each file and periodically test them? I have a couple of big drives for family media I'd love to protect with a bit more assurance than "the drive did not fail".

Yes, zfs includes file-level checksums.

Re: My 71 TiB ZFS NAS After 10 Years and Zero Drive Failures

#73

Post by ghostly_s »

foobarian wrote: post #69
turnsout wrote: post #8

I’ve heard the exact opposite advice (keep the drives running to reduce wear from power cycling). Not sure what to believe, but I like having my ZFS NAS running so it can regularly run scrubs and check the data. FWIW, I’ve run my 4 drive system for 10 years with 2 drive failures in that time, but they were not enterprise grade drives (WD Green).

> regularly run scrubs and check the data Does this include some kind of built-in hash/checksum system to record e.g. md5 sums of each file and periodically test them? I have a couple of big drives for family media I'd love to protect with a bit more assurance than "the drive did not fail".

https://en.wikipedia.org/wiki/ZFS?wprov=sfti1#Resilvering_an...

Re: My 71 TiB ZFS NAS After 10 Years and Zero Drive Failures

#74

Post by Filligree »

Jedd wrote: post #42

Surprised to not find 'ecc' on that page. I know it's not a guarantee of no-corruption, and ZFS without ECC is probably no more dangerous than any other file system without ECC, but if data corruption is a major concern for you, and you're building out a pretty hefty system like this, I can't imagine not using ECC. Slow on-disk data corruption resulting from gradual and near-silent RAM failures may be like doing regu…

It's difficult as a home user to find ECC memory, harder to make sure it actually works in your hardware configuration, and near-impossible to find ECC memory that doesn't require lower speeds than what you can get for $50 on amazon.

I would very much like to put ECC memory in my home server, but I couldn't figure it out this generation. After four hours I decided I had better things to do with my time.

Re: My 71 TiB ZFS NAS After 10 Years and Zero Drive Failures

#75

Post by tomxor »

Dalewyn wrote: post #47
turnsout wrote: post #8

I’ve heard the exact opposite advice (keep the drives running to reduce wear from power cycling). Not sure what to believe, but I like having my ZFS NAS running so it can regularly run scrubs and check the data. FWIW, I’ve run my 4 drive system for 10 years with 2 drive failures in that time, but they were not enterprise grade drives (WD Green).

>Not sure what to believe Keep them running. Why?: * The read/write heads experience literally next to no wear while they are floating above the platters. They physically land onto shelves or onto landing zones on the platters themselves when turned off; landing and takeoff are by far the most wear the heads will suffer. * Following on the above, in the worst case the read/write heads might be torn off during takeoff…

> Keep them running [...] Bearings will last longer; they might also seize up if left stationary for too long. Likewise the drive motor.

All HDD failures I've ever seen in person (5 across 3 decades), were bearing failures, in machine that were almost always on with drives spun up. It's difficult to know for sure without proper A-B comparisons, but I've never seen a bearing failure in a machine where drives were spun down automatically.

It also seems intuitive that for mechanical bearings the longer they are spun up the greater the wear and the greater the chance of failure.

Re: My 71 TiB ZFS NAS After 10 Years and Zero Drive Failures

#77

Post by Cyph0n »

foobarian wrote: post #69
turnsout wrote: post #8

I’ve heard the exact opposite advice (keep the drives running to reduce wear from power cycling). Not sure what to believe, but I like having my ZFS NAS running so it can regularly run scrubs and check the data. FWIW, I’ve run my 4 drive system for 10 years with 2 drive failures in that time, but they were not enterprise grade drives (WD Green).

> regularly run scrubs and check the data Does this include some kind of built-in hash/checksum system to record e.g. md5 sums of each file and periodically test them? I have a couple of big drives for family media I'd love to protect with a bit more assurance than "the drive did not fail".

Yep, ZFS reads everything in the array and validates checksums. ZFS (at least on Linux) ships with scrub systemd timers: https://openzfs.github.io/openzfs-docs/man/master/8/zpool-sc...

Re: My 71 TiB ZFS NAS After 10 Years and Zero Drive Failures

#78

Post by telgareith »

Dalewyn wrote: post #47
turnsout wrote: post #8

I’ve heard the exact opposite advice (keep the drives running to reduce wear from power cycling). Not sure what to believe, but I like having my ZFS NAS running so it can regularly run scrubs and check the data. FWIW, I’ve run my 4 drive system for 10 years with 2 drive failures in that time, but they were not enterprise grade drives (WD Green).

>Not sure what to believe Keep them running. Why?: * The read/write heads experience literally next to no wear while they are floating above the platters. They physically land onto shelves or onto landing zones on the platters themselves when turned off; landing and takeoff are by far the most wear the heads will suffer. * Following on the above, in the worst case the read/write heads might be torn off during takeoff…

Citations needed.

Counterpoints for each: Heads don't suffer wear when parking. The armature does.

If the platters are not spinning fast enough, or the air density is too low: the heads will crash into the sides of the platters.

The main wear on platter bearings is vibration, it takes an extremely long time for the lube to "gum up." If its still a thing at all. I suspect it used to happen because they were petroleum distilate lubes. So, shorter chains would evaporate/sublimate leaving longer more viscous chains. Or straight polymerize.

With fully synthetic PAO oils, and other options they won't do that anymore.

What inrush? They're polyphase steppers. The only reason for inrush is that the engineers didn't think it'd affect lifetime.

Counter: turn your drives off, thebsaved power of 8 drives being off half the day easily totals $80 a year- enough to replace all but the highest capacities.

Re: My 71 TiB ZFS NAS After 10 Years and Zero Drive Failures

#79

Post by InvaderFizz »

rincebrain wrote: post #66
louwrentius wrote: post #46

Earlier quoted context omitted.

The system is using ECC and I specifically - unrelated to ZFS - wanted to use ECC memory to reduce risk of data/fs corruption. I've also added 'ecc' to the original blog post to clarify. Edit: ZFS for home usage doesn't need a ton of RAM as far as I've learned. There is the 1 GB of RAM per 1TB of storage rule of thumb, but that was for a specific context. Maybe the ill-fated data deduplication feature, or was it just…

It was a handwavey rule of estimation for dedup, handwavey because dedup scales on number of records, which is going to vary wildly by recordsize.

Additionally unless it's changed in the last six years, you should pretend ZFS dedupe doesn't exist.

Re: My 71 TiB ZFS NAS After 10 Years and Zero Drive Failures

#80

Post by stavros »

rnxrx wrote: post #59

In my experience the environment where the drives are running makes a huge difference in longevity. There's a ton more variability in residential contexts than in data center (or even office) space. Potential temperature and humidity variability is a notable challenge but what surprised me was the marked effect of even small amounts of dust. Many years ago I was running an 8x500G array in an old Dell server in my bas…

How does dust affect things? The drives are airtight.
Post reply on HN