Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

My 71 TiB ZFS NAS After 10 Years and Zero Drive Failures

louwrentius.com

Post reply on HN
Print view View original

201–210 of 314 posts

Re: My 71 TiB ZFS NAS After 10 Years and Zero Drive Failures

#201

Post by homebrewer »

clan wrote: post #163
manuel_w wrote: post #160

Discussions on checksumming filesystems usually revolve around ZFS and BTRFS, but has someone any experience with bcachefs? It's upstreamed in the linux kernel, I learned, and is supposed to have full checksumming. The author also seems to take filesystem responsibility seriously. Is anyone using it around here? https://bcachefs.org/

That was a decision Linus regretted[1]. There has been some recent discussion about this here on Hacker News[2]. [1] https://linuxiac.com/torvalds-expresses-regret-over-merging-... [2] https://news.ycombinator.com/item?id=41407768

As usual, the top comments in that submission are very biased. I think HN should sort comments in a random order in every polarizing discussion. Anyone reading this, do yourself a favor and dig through both links, or ignore the parent's comment altogether.

Linus "regretted" it in the sense "it was a bit too early because bcachefs is moving at such a fast speed", and not in the sense "we got a second btrfs that eats your data for lunch".

Please provide context and/or short human-friendly explanation, because I'm pretty sure most readers won't go further than your comment and will remember it as "Linus regrets merging bcachefs", helping spread FUD for years down the line.

Re: My 71 TiB ZFS NAS After 10 Years and Zero Drive Failures

#202

Post by rollcat »

n3storm wrote: post #196

After 20 years using ext3 and ext4 I only lost data when ffing around with parted and did.

There's an infinite amount of ways you can lose data. Here's one of my recent stories: https://www.rollc.at/posts/2022-05-02-disaster-recovery/

Just among my "personal" stuff, over the last 12 years I've completely lost 4 hard drives due to age/physical failure. ZFS made it a no-deal twice, and aided with recovery once (once I've dd'd what was left of one drive, zvol snapshots made "risky" experimentation cheap&easy).

Re: My 71 TiB ZFS NAS After 10 Years and Zero Drive Failures

#203

Post by patchymcnoodles »

jiggawatts wrote: post #52
patchymcnoodles wrote: post #24

Earlier quoted context omitted.

Exactly that. I'm not even shooting in ProRes or similar "raw" video. But one video project easily takes 3TB. And I'm not even a professional.

Holy cow, what are you shooting with!? I have a Nikon Z8 that can output up to 8.3K @ 60 fps raw video, and my biggest project is just 1 TB! Most are on the order of 20 GB, if that.

I use a Sony a1, my videos are 4k 100fps. But on the last project I also had an Insta 360 x4 shooting B-Roll. So on some days that adds up a lot.

Re: My 71 TiB ZFS NAS After 10 Years and Zero Drive Failures

#204

Post by rollcat »

manuel_w wrote: post #160

Discussions on checksumming filesystems usually revolve around ZFS and BTRFS, but has someone any experience with bcachefs? It's upstreamed in the linux kernel, I learned, and is supposed to have full checksumming. The author also seems to take filesystem responsibility seriously. Is anyone using it around here? https://bcachefs.org/

Can't comment on bcachefs (I think it's still early), but I've been running with bcache in production on one "canary" machine for years, and it's been rock-solid.

Re: My 71 TiB ZFS NAS After 10 Years and Zero Drive Failures

#205

Post by KMag »

homebrewer wrote: post #195
KMag wrote: post #167

Earlier quoted context omitted.

But if you have a single board computer with 1 GB of RAM and several TB of ZFS, will it just be slow, or actually not run? Granted, my use case was abnormal, and I was evaluating in the early days when there were both license and quality concerns with ZFS on Linux. However, my understanding at the time was that it wouldn't actually work to have several TB in a ZFS pool with 1 GB of RAM. My understanding is that ZFS h…

> will it just be slow This. I use it on a tiny backup server with only 1 GB of RAM and a 4 TB HDD pool, it's fine. Only one machine backs up to that server at a time, and they do that at network speed (which is admittedly only 100 Mb/s, but it should go somewhat higher if it had faster network). Restore also runs ok.

Thanks for this. I initially went with xfs back when there were license and quality concerns with zfs on Linux before btrfs was a thing, and moved to btrfs after btrfs was created and matured a bit.

These days, I think I would be happier with zfs and one RAID-Z pool across all of the disks instead of individual btrfs partitions or btrfs on RAID 5.

Re: My 71 TiB ZFS NAS After 10 Years and Zero Drive Failures

#206

Post by hulitu »

mustache_kimono wrote: post #105
snvzz wrote: post #76

>but for residential usage, it's totally reasonable to accept the risk. Polite disagree. Data integrity is the natural expectation humans have from computers, and thus we should stick to filesystems with data checksums such as ZFS, as well as ECC memory.

> Data integrity is the natural expectation humans have from computers I've said it once, and I'll say it again: the only reason ZFS isn't the norm is because we all once lived through a primordial era when it didn't exist. No serious person designing a filesystem today would say it's okay to misplace your data. Not long ago, on this forum, someone told me that ZFS is only good because it had no competitors in its sp…

> the only reason ZFS isn't the norm is because we all once lived through a primordial era when it didn't exist.

There were good filesystems before ZFS. I would love to have a versioning filesystem like Apollo had.

Re: My 71 TiB ZFS NAS After 10 Years and Zero Drive Failures

#207

Post by Gud »

bboygravity wrote: post #133
Gud wrote: post #130

Earlier quoted context omitted.

Electronics is absolutely sensitive to this. Please use filters.

Filters won't help against prolonged periods of higher/lower voltages though.

Voltages should be normalised before they hit the servers psu.

Re: My 71 TiB ZFS NAS After 10 Years and Zero Drive Failures

#208

Post by magicalhippo »

doublepg23 wrote: post #152
KMag wrote: post #150

Earlier quoted context omitted.

How are the memory overheads of ZFS these days? In the old days, I remember balking at the extra memory required to run ZFS on the little ARM board I was using for a NAS.

That was always FUD more or less. ZFS uses RAM as its primary cache…like every other filesystem, so it if you have very little RAM for caching the performance will degrade…like every other filesystem.

> That was always FUD more or less.

To give some context. ZFS support de-duplication, and until fairly recently, the de-duplication data structures had to be resident in memory.

So if you used de-duplication earlier, then yes, you absolutely did need a certain amount of memory per byte stored.

However, there is absolutely no requirement to use de-duplication, and without it the memory requirements are just a small, fairly fixed amount.

It'll store writes in memory until it commits them in a so-called transaction group, so you need to have room for that. But the limits on a transaction group is configurable, so you can lower the defaults.

Re: My 71 TiB ZFS NAS After 10 Years and Zero Drive Failures

#209

Post by ffsm8 »

manuel_w wrote: post #160

Discussions on checksumming filesystems usually revolve around ZFS and BTRFS, but has someone any experience with bcachefs? It's upstreamed in the linux kernel, I learned, and is supposed to have full checksumming. The author also seems to take filesystem responsibility seriously. Is anyone using it around here? https://bcachefs.org/

I tried it out on my homelab server right after the merge into the Linux kernel.

Took roughly one week for the whole raid to stop mounting because of the journal (8hdd, 2 ssd write cache, 2 nvme read cache).

The author responded on Reddit within a day, I tried his fix, (which meant compiling the Linux kernel and booting from that), but his fix didn't resolve the issue. He sadly didn't respond after that, so I wiped and switched back to a plain mdadmin raid after a few days of waiting.

I had everything important backed up, obviously (though I did lose some unimportant data), but it did remind me that bleeding edge is indeed ... Unstable

The setup process and features are fantastic however, simply being able to add a disk and flag it as read/write cache feels great. I'm certain I'll give it another try in a few years, after it had some time in the oven.

Re: My 71 TiB ZFS NAS After 10 Years and Zero Drive Failures

#210

Post by louwrentius »

BSDobelix wrote: post #184
louwrentius wrote: post #178

Earlier quoted context omitted.

Most people don’t run ZFS on their laptop, desktop so let’s not pretend it’s such a huge deal.

Most people run Windows on their Laptop (without ReFS), and many people use paid data restore services if something "important" gets missing/corrupt. >let’s not pretend it’s such a huge deal Depends on the importance of you data right?

I bet even 99.9% of HN visitors don’t run ZFS on their laptop/desktop. Basically we all take this risk except for a few dedicated nerds.

Everything has a price and people like to have their uncorrupted files, but not at all cost.

Post reply on HN