In my experience the environment where the drives are running makes a huge difference in longevity. There's a ton more variability in residential contexts than in data center (or even office) space. Potential temperature and humidity variability is a notable challenge but what surprised me was the marked effect of even small amounts of dust. Many years ago I was running an 8x500G array in an old Dell server in my bas…
My 71 TiB ZFS NAS After 10 Years and Zero Drive Failures
81–90 of 314 posts
Re: My 71 TiB ZFS NAS After 10 Years and Zero Drive Failures
#82In my experience the environment where the drives are running makes a huge difference in longevity. There's a ton more variability in residential contexts than in data center (or even office) space. Potential temperature and humidity variability is a notable challenge but what surprised me was the marked effect of even small amounts of dust. Many years ago I was running an 8x500G array in an old Dell server in my bas…
How does dust affect things? The drives are airtight.
Re: My 71 TiB ZFS NAS After 10 Years and Zero Drive Failures
#83edit: looks like for the netherlands (where he lives) this is more significant -- $0.50/kWh is the average price, so ~$32/year
Re: My 71 TiB ZFS NAS After 10 Years and Zero Drive Failures
#84In my experience the environment where the drives are running makes a huge difference in longevity. There's a ton more variability in residential contexts than in data center (or even office) space. Potential temperature and humidity variability is a notable challenge but what surprised me was the marked effect of even small amounts of dust. Many years ago I was running an 8x500G array in an old Dell server in my bas…
Hah! I had a 16x500GB Seagate array and also averaged an RMA every six months. I think there was a firmware issue with that generation.
Re: My 71 TiB ZFS NAS After 10 Years and Zero Drive Failures
#85I'm confused about optimizing 7 watts as important -- rough numbers, 7 watts is 61 kWh/y. If you assume US-average prices of $0.16/kWh that's about $10/year. edit: looks like for the netherlands (where he lives) this is more significant -- $0.50/kWh is the average price, so ~$32/year
Where in US matters a lot. I think OR is pretty cheap.
Re: My 71 TiB ZFS NAS After 10 Years and Zero Drive Failures
#86In my experience the environment where the drives are running makes a huge difference in longevity. There's a ton more variability in residential contexts than in data center (or even office) space. Potential temperature and humidity variability is a notable challenge but what surprised me was the marked effect of even small amounts of dust. Many years ago I was running an 8x500G array in an old Dell server in my bas…
Re: My 71 TiB ZFS NAS After 10 Years and Zero Drive Failures
#87Earlier quoted context omitted.
>Not sure what to believe Keep them running. Why?: * The read/write heads experience literally next to no wear while they are floating above the platters. They physically land onto shelves or onto landing zones on the platters themselves when turned off; landing and takeoff are by far the most wear the heads will suffer. * Following on the above, in the worst case the read/write heads might be torn off during takeoff…
> Keep them running [...] Bearings will last longer; they might also seize up if left stationary for too long. Likewise the drive motor. All HDD failures I've ever seen in person (5 across 3 decades), were bearing failures, in machine that were almost always on with drives spun up. It's difficult to know for sure without proper A-B comparisons, but I've never seen a bearing failure in a machine where drives were spun…
Re: My 71 TiB ZFS NAS After 10 Years and Zero Drive Failures
#88Surprised to not find 'ecc' on that page. I know it's not a guarantee of no-corruption, and ZFS without ECC is probably no more dangerous than any other file system without ECC, but if data corruption is a major concern for you, and you're building out a pretty hefty system like this, I can't imagine not using ECC. Slow on-disk data corruption resulting from gradual and near-silent RAM failures may be like doing regu…
The system is using ECC and I specifically - unrelated to ZFS - wanted to use ECC memory to reduce risk of data/fs corruption. I've also added 'ecc' to the original blog post to clarify. Edit: ZFS for home usage doesn't need a ton of RAM as far as I've learned. There is the 1 GB of RAM per 1TB of storage rule of thumb, but that was for a specific context. Maybe the ill-fated data deduplication feature, or was it just…
I'd heard the original ZFS memory estimations were somewhat exuberant, and recommendations had come down a lot since the early days, but I'd imagine given your usage pattern - powered on periodically - a performance hit for whatever operations you're doing during that time wouldn't be problematic.
I used to use mdadm for software RAID, but for several years now my home boxes are all hardware RAID. LVM2 provides the other features I need, so I haven't really ever explored zfs as a replacement for both - though everyone I know that uses it, loves it.
Re: My 71 TiB ZFS NAS After 10 Years and Zero Drive Failures
#89Surprised to not find 'ecc' on that page. I know it's not a guarantee of no-corruption, and ZFS without ECC is probably no more dangerous than any other file system without ECC, but if data corruption is a major concern for you, and you're building out a pretty hefty system like this, I can't imagine not using ECC. Slow on-disk data corruption resulting from gradual and near-silent RAM failures may be like doing regu…
It's difficult as a home user to find ECC memory, harder to make sure it actually works in your hardware configuration, and near-impossible to find ECC memory that doesn't require lower speeds than what you can get for $50 on amazon. I would very much like to put ECC memory in my home server, but I couldn't figure it out this generation. After four hours I decided I had better things to do with my time.
A decade ago if you wanted ECC your choice was basically Xeon, and all() Xeon motherboards would accept ECC.
I agree that these days it's much more complex, since you are ineluctably going get sucked into the despair-spiral of trying to work out what combination of Ryzen + motherboard + ECC RAM will give you actual, demonstrable* ECC (with correction, not just detection).
Re: My 71 TiB ZFS NAS After 10 Years and Zero Drive Failures
#90Earlier quoted context omitted.
> Keep them running [...] Bearings will last longer; they might also seize up if left stationary for too long. Likewise the drive motor. All HDD failures I've ever seen in person (5 across 3 decades), were bearing failures, in machine that were almost always on with drives spun up. It's difficult to know for sure without proper A-B comparisons, but I've never seen a bearing failure in a machine where drives were spun…
I think I have lost half a dozen hard drives (and a couple DVD-RW drives) over the decades because they sat in a box for a couple years on a shelf (I recall that one recovered working with a higher amperage 12V supply, but only long enough to copy off most of the data)