Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

My 71 TiB ZFS NAS After 10 Years and Zero Drive Failures

louwrentius.com

Post reply on HN
Print view View original

231–240 of 314 posts

Re: My 71 TiB ZFS NAS After 10 Years and Zero Drive Failures

#231

Post by lazide »

londons_explore wrote: post #200
orbital-decay wrote: post #111

Do you have a drive rotation schedule? 24 drives. Same model. Likely the same batch. Similar wear. Imagine most of them failing at the same time, and the rest failing as you're rebuilding it due to the increased load, because they're already almost at the same point. Reliable storage is tricky.

Software bugs might cause that (eg. drive fails after exactly 1 billion IOPS due to some counter overflowing). But hardware wear probably won't be as consistent.

That depends entirely on how good their Q&A and manufacturing quality is - the better it is, the more likely eh?

Especially in an array where it’s possible every drive operation will be identical between 2 or 3 different drives.

Re: My 71 TiB ZFS NAS After 10 Years and Zero Drive Failures

#232

Post by ragebol »

louwrentius wrote: post #180
ffsm8 wrote: post #129

> Losing the system due to power shenanigans is a risk I accept. There is another (very rare) failure an ups protects against, and that's imbalance in the electricity. You can get a spike (up or down, both can be destructive) if there is construction in your area and something happens with the electricity, or lightning hits a pylon close enough to your house. First job I worked at had multiple servers die like that,…

True, this is also what I mean with power shenanigans. My server is off most off the time, disconnected. But even if it wasn’t, I just accept the risk.

Assuming you live in the Netherlands judging just by name: our power grid is pretty damn reliable with little shenanigans. I'd take that risk indeed.

Re: My 71 TiB ZFS NAS After 10 Years and Zero Drive Failures

#233

Post by foepys »

Tepix wrote: post #226

Having 24 drives probably offers some performance advantages, but if you don‘t require them, having a 6-bay NAS with 18TB disks instead woukld offer a ton of advantages in terms of power usage, noise, space required, cost and reliability.

I'd want more redundancy in that case. With such large HDDs zfs resilver could kill another disk and then you would lose your data.

Re: My 71 TiB ZFS NAS After 10 Years and Zero Drive Failures

#234

Post by simoncion »

curt15 wrote: post #229
simoncion wrote: post #219

Earlier quoted context omitted.

> I never had the guts to try btrfs in production after all the horror stories I've heard over the decade+. I've been running btrfs as the primary filesystem for all of my desktop machines since shortly after the on-disk format stabilized and the extX->btrfs in-place converter appeared [0], and for my home servers for the past ~five years. In the first few years after I started using it on my desktop machines, I had…

>At $DAYJOB we use btrfs as the filesystem for our CI workers and have been doing so for years. Its snapshot functionality makes creating the containers for CI jobs instantaneous, and we've had zero problems with it. $DAYJOB == "facebook"?

> $DAYJOB == "facebook"?

Nah. AFAIK, we don't have any Linux kernel gurus on the payroll... we're ordinary users just like most everyone else.

Re: My 71 TiB ZFS NAS After 10 Years and Zero Drive Failures

#236

Post by prepend »

Tepix wrote: post #226

Having 24 drives probably offers some performance advantages, but if you don‘t require them, having a 6-bay NAS with 18TB disks instead woukld offer a ton of advantages in terms of power usage, noise, space required, cost and reliability.

18TB drives didn’t exist back when this setup was designed.

Of course a 3 bay with 128TB drives would also be superior, but this comment only makes sense in a few years.

Re: My 71 TiB ZFS NAS After 10 Years and Zero Drive Failures

#237

Post by rollcat »

nubinetwork wrote: post #225
rollcat wrote: post #223

Earlier quoted context omitted.

> So modify the ZFS source Way to miss the point

Not really... the complaint was over licensing and tainting the kernel... so just tell the kernel it's not a CDDL module... problem solved.

> What are the practical choices for people who do not want to cosplay as sysadmins?

The specific complaint is not at all about the kernel identifying itself as tainted, the specific complaint is about the kernel developers' unyielding unwillingness to support any scenario where ZFS is concerned, thus leaving one with even more "sysadmin duties". I want to use my computer, not serve it.

Re: My 71 TiB ZFS NAS After 10 Years and Zero Drive Failures

#238

Post by Novosell »

homebrewer wrote: post #201
clan wrote: post #163

Earlier quoted context omitted.

That was a decision Linus regretted[1]. There has been some recent discussion about this here on Hacker News[2]. [1] https://linuxiac.com/torvalds-expresses-regret-over-merging-... [2] https://news.ycombinator.com/item?id=41407768

As usual, the top comments in that submission are very biased. I think HN should sort comments in a random order in every polarizing discussion. Anyone reading this, do yourself a favor and dig through both links, or ignore the parent's comment altogether. Linus "regretted" it in the sense "it was a bit too early because bcachefs is moving at such a fast speed", and not in the sense "we got a second btrfs that eats y…

You're saying this like the takeaway of "Linus regrets merging bcachefs" is unfair when the literal quote from Linus is "[...] I'm starting to regret merging bcachefs." And earlier he says "Nobody sane uses bcachefs and expects it to be stable[...]".

I don't understand how you can read Linus' response and think "Linus regrets merging bcachefs" is an unfair assessment.

Re: My 71 TiB ZFS NAS After 10 Years and Zero Drive Failures

#239

Post by not_the_fda »

prepend wrote: post #235

I wish he had talked more about his movie collection. I’m interested in the methods of selecting initial items as well as ones that survive in the collection for 10+ years.

My server isn't nearly as big as his, but my collection is mostly Criterion https://www.criterion.com/closet-picks

Re: My 71 TiB ZFS NAS After 10 Years and Zero Drive Failures

#240

Post by iforgotpassword »

ffsm8 wrote: post #209
manuel_w wrote: post #160

Discussions on checksumming filesystems usually revolve around ZFS and BTRFS, but has someone any experience with bcachefs? It's upstreamed in the linux kernel, I learned, and is supposed to have full checksumming. The author also seems to take filesystem responsibility seriously. Is anyone using it around here? https://bcachefs.org/

I tried it out on my homelab server right after the merge into the Linux kernel. Took roughly one week for the whole raid to stop mounting because of the journal (8hdd, 2 ssd write cache, 2 nvme read cache). The author responded on Reddit within a day, I tried his fix, (which meant compiling the Linux kernel and booting from that), but his fix didn't resolve the issue. He sadly didn't respond after that, so I wiped a…

New filesystems seems to have a chicken and egg problem really. It's not like switching from Nvidia's proprietary drivers to nouveau and then back if it turns out they don't work that well. Switching filesystems, especially in larger raid setups where you desperately need more testing and real world usage feedback, is pretty involved, and even if you have everything backed up it's pretty time consuming restoring everything should things go haywire.

And even if you have the time and patience to be one of these early adopters, debugging any issues encountered might also be difficult, as ideally you want to give the devs full access to your filesystem for debugging and attempted fixes, which is obviously not always feasible.

So anything beyond the most trivial setups and usage patterns gets a miniscule amount of testing.

In an ideal world, you'd nail your FS design first try, make no mistakes during implementation and call it a day. I'd like to live in an ideal world.

Post reply on HN