Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

My 71 TiB ZFS NAS After 10 Years and Zero Drive Failures

louwrentius.com

Post reply on HN
Print view View original

181–190 of 314 posts

Re: My 71 TiB ZFS NAS After 10 Years and Zero Drive Failures

#181

Post by taneq »

AndrewDavis wrote: post #121
naming_the_user wrote: post #100

Earlier quoted context omitted.

I agree. I think that the author may not have experienced these sorts of errors before. Yes, the average person may not care about experiencing a couple of bit flips per year and losing the odd pixel or block of a JPEG, but they will care if some cable somewhere or transfer or bad RAM chip or whatever else manages to destroy a significant amount of data before they notice it.

I had a significant data loss years ago. I was young and only had a deskop, so all my data was there. So I purchased a 300GB external usb drive to use for periodic backup. It was all manual copy/paste files across with no real schedule, but it was fine for the time and life was good. Over time my data grew and the 300GB drive wasn't large enough to store it all. For a while some of it wasnt backed up (I was young wit…

My last spinning hard drive failed silently like this, but I didn’t lose too much data… I think. The worst part is not knowing.

Re: My 71 TiB ZFS NAS After 10 Years and Zero Drive Failures

#182

Post by mytailorisrich »

snvzz wrote: post #76

>but for residential usage, it's totally reasonable to accept the risk. Polite disagree. Data integrity is the natural expectation humans have from computers, and thus we should stick to filesystems with data checksums such as ZFS, as well as ECC memory.

I think this is over the top for standard residential usage.

Make sure you have good continuous backups and perhaps RAID 1 on your file server (if you have one) to save efforts in case a disk fails and you are more than covered.

Re: My 71 TiB ZFS NAS After 10 Years and Zero Drive Failures

#183

Post by louwrentius »

fulafel wrote: post #120

Regular reminder: RAID (and ZFS) don't replace backups. It's an availability solution to reduce downtime in event of disk failure. Many things can go wrong with your files and filesystem besides disk failure, eg user error, userspace software/script bugs, driver or FS or hardware bugs, ransomware, etc) The article mentions backups near the end saying eg "most of the data is not important" and the "most important" dat…

I’m well aware of the risks, I just accept it.

You shouldn’t ever do what I do if you really care about tour data.

Re: My 71 TiB ZFS NAS After 10 Years and Zero Drive Failures

#184

Post by BSDobelix »

louwrentius wrote: post #178
snvzz wrote: post #76

>but for residential usage, it's totally reasonable to accept the risk. Polite disagree. Data integrity is the natural expectation humans have from computers, and thus we should stick to filesystems with data checksums such as ZFS, as well as ECC memory.

Most people don’t run ZFS on their laptop, desktop so let’s not pretend it’s such a huge deal.

Most people run Windows on their Laptop (without ReFS), and many people use paid data restore services if something "important" gets missing/corrupt.

>let’s not pretend it’s such a huge deal

Depends on the importance of you data right?

Re: My 71 TiB ZFS NAS After 10 Years and Zero Drive Failures

#185

Post by louwrentius »

why_only_15 wrote: post #83

I'm confused about optimizing 7 watts as important -- rough numbers, 7 watts is 61 kWh/y. If you assume US-average prices of $0.16/kWh that's about $10/year. edit: looks like for the netherlands (where he lives) this is more significant -- $0.50/kWh is the average price, so ~$32/year

Althoug outside the scope of the artice, I try to keep a small electricity usage footprint.

My lowest is 85Kwh in an apartment a moth but that was because of perfect solar weather.

I average around 130 KWH a month now.

Re: My 71 TiB ZFS NAS After 10 Years and Zero Drive Failures

#186

Post by louwrentius »

tedk-42 wrote: post #114

Really a non article as it feels like an edge case for usage. It's not on 24/7. No mention of I/O metrics or data stored. For all we know, OP is storing their photos and videos and never actually need to have 80% of the drives actually on and connected.

It’s 80% full. I linked to the original article about the system for perf stats (sequential)

Re: My 71 TiB ZFS NAS After 10 Years and Zero Drive Failures

#187

Post by BSDobelix »

doublepg23 wrote: post #152
KMag wrote: post #150

Earlier quoted context omitted.

How are the memory overheads of ZFS these days? In the old days, I remember balking at the extra memory required to run ZFS on the little ARM board I was using for a NAS.

That was always FUD more or less. ZFS uses RAM as its primary cache…like every other filesystem, so it if you have very little RAM for caching the performance will degrade…like every other filesystem.

>That was always FUD more or less

Thank you thank you, exactly this! And additionally that cache is compressed. In the day's of 4GB machines ZFS was overkill but today...no problem.

Re: My 71 TiB ZFS NAS After 10 Years and Zero Drive Failures

#188

Post by mrjin »

nolok wrote: post #147
Timshel wrote: post #143

Earlier quoted context omitted.

Just checked my scrub history, for 20TB on consumer hardware during the last two years it repaired twice around 2 and 4 blocks each time. So not much but at the same time with a special kind of luck might have been on an encrypted archive ^^.

That's all fine and good until that one random lone broken block stops you from opening that file you really need.

What you need is backup. RAID is not backup and is not for most home/personal users. I learnt that the hard way. Now my NAS use simple volumes only, after all, I really don't have many things I cannot lose on it. If it's something really important, I have multiple copies on different drives, and some offline cold backup. So now if any of my NAS drive is about to fail, I can just copy out the data and replace the drive, instead of spending weeks trying to rebuild the RAID and ended with a total loss as multiple drives failed in a row. The funny thing is that, after moving to simply volumes approach, I never had a drive with even a bad sector since.

Re: My 71 TiB ZFS NAS After 10 Years and Zero Drive Failures

#190

Post by louwrentius »

matheusmoreira wrote: post #107

> It's possible to create the same amount of redundant storage space with only 6-8 hard drives with RAIDZ2 (RAID 6) redundancy. I've given up on striped RAID. Residential use requires easy expandability to keep costs down. Expanding an existing parity stripe RAID setup involves failing every drive and slowly replacing them one by one with bigger capacity drives while the whole array is in a degraded state and incurri…

I dislike that article/advice because it’s dishonest / downplaying a limitation of ZFS and advocating that people should spend a lot more money, that may likely not be necessary at all.
Post reply on HN