ZFS fans, rejoice – RAIDz expansion will be a thing soon • Hacker News Board

Re: ZFS fans, rejoice – RAIDz expansion will be a thing soon

#171

Post by rektide » Sat, Jun 19, 2021, 2:46 PM UTC

nix23 wrote: post #168

rektide wrote: post #130

Earlier quoted context omitted.

Snapshots & send-receive are my favorite BTRFS features too! I too love snapshotting Debian installs & send'ing them to new machines for atomic updates, or to build live usb images. I've been using Pottering's "Revisiting How We Put Together Linux Systems" naming system for my subvolumes, and that's worked fairly well[1]. I even did some fiddling around with enhancing Debian-live to load btrfs into ram a while back,…

>we should be cooperative & boosting each other. Is that why you sound like a sour apple?

I'm sour because it feels like zfs folks in particular have it in for btrfs and it's very tiring. there's so many OSes which default to btrfs that there's be some real modern evidence if there were problems but we are constantly dogged by massive negativity.

Re: ZFS fans, rejoice – RAIDz expansion will be a thing soon

#172

Post by WastingMyTime89 » Sat, Jun 19, 2021, 3:09 PM UTC

nix23 wrote: post #166

WastingMyTime89 wrote: post #160

Earlier quoted context omitted.

> Meanwhile ZFS is far far better than the alternatives (btrfs) in terms of data integrity and reliability. No, it's not. Btrfs is nowadays perfectly reliable as long as you avoid the in development features which are all explicitly marked as in development and will warn you. That's why Facebook uses it in production. It also has some nice advantage for home use. Btrfs pools are a lot simpler to manage and have less…

I used/use SLES for a long time, and there is a strong recommendation from SUSE itself how you should btrfs with SLES. -Just use it for the OS, XFS for data -Just use it in mirror configuration -Don't touch anything else -Make Snapshots and delete them is stable With those points, i too had never a problem.

Novell controls the XFS development and Redhat has bought a competing technology. Obviously they are not going to push Btrfs. Its use amongst large companies speak for itself however.

I see that tge ZFS fanboys are out in force however. I mean, people should be free to use a second class citizen zoth a terrible license and a pool gestion made for masochists if they want.

Re: ZFS fans, rejoice – RAIDz expansion will be a thing soon

#173

Post by dobsonj » Sat, Jun 19, 2021, 3:17 PM UTC

rektide wrote: post #78

nimbius wrote: post #53

this might sound like a troll comment but its coming from someone with almost zero experience with raid. What is the purpose of ZFS in 2021 if we have hardware RAID and linux software RAID? BTRFS does RAID too. Why would people choose ZFS in 2021 if both Oracle and Open Source users have 2 competing ZFS? are they interoperable?

No matter what happens, people will seemingly forever declare BTRFS is not as stable and not as safe. There's a status page that details what BTRFS thinks of itself[1], and I doubt any of the many people docking BTRFS have read or know or care what that page says. There is one issue still being worked out to completion, a "write hole" problem, involving two separate failures, an unplanned/power-loss shut-down, follow…

> BTRFS has proven remarkably stable for a while now. A decade ago that wasn't quite as absolutely bulletproof, but today the situation is much different.

When I did a fresh install of Fedora 33 on my primary workstation 3 months ago, I had exactly the same rationale for sticking with the default BTRFS selection. "I'm sure it has come quite a long way since I last tried it, I would like to have some of those features, I know there are some large production installations now, and the fact that it's the default in Fedora is a sign of confidence from the community."

After 2 months of use, I ended up with a corrupt filesystem in the middle of my work day and could not find any way to recover from it other than to do a full reinstall and restore my files from a backup. This was on a single NVMe drive in a system running a few small VM's, a browser, a chat client, and a few terminals. Thankfully I only lost a day's worth of work, but that's the last time I install BTRFS on any of my personal systems.

> [...] it feels a bit odd that BTRFS is such a persistent target of slander & assault.

My experience is obviously anecdotal (as are all individual experiences), so I won't be surprised if you dismiss my comment like you did nullwarp's comment. But "slander & assault" just seems like a weird way to dismiss all critics of BTRFS at once, as if everyone is out to get BTRFS. Filesystems have a thankless job. Do it right, and most users will never even think about it. But lose a user's data once, and you've likely lost that user forever.

> A number of big names use BTRFS, including Facebook. I have yet to see any hyperscalers interested in ZFS.

ZFS is excellent for large arrays of spinning disks, but if you're using a bunch of fast SSD's, performance really sucks. There is a lot of lock contention contributing to that which isn't noticeable on slower devices. I can't speak to FB's environment, but if they're managing a large number of SSD's like most hyperscalers, then ZFS would probably get ruled out based on performance comparisons.

Re: ZFS fans, rejoice – RAIDz expansion will be a thing soon

#174

Post by michaelmrose » Sat, Jun 19, 2021, 4:03 PM UTC

WastingMyTime89 wrote: post #172

nix23 wrote: post #166

Earlier quoted context omitted.

I used/use SLES for a long time, and there is a strong recommendation from SUSE itself how you should btrfs with SLES. -Just use it for the OS, XFS for data -Just use it in mirror configuration -Don't touch anything else -Make Snapshots and delete them is stable With those points, i too had never a problem.

Novell controls the XFS development and Redhat has bought a competing technology. Obviously they are not going to push Btrfs. Its use amongst large companies speak for itself however. I see that tge ZFS fanboys are out in force however. I mean, people should be free to use a second class citizen zoth a terrible license and a pool gestion made for masochists if they want.

Why is Novell's statement that BTRFS is only narrowly useful trivially ignored but large companies eg Facebook using it meant to be interpreted as positive proof while handily ignoring that Facebook is using it for ephemeral data.

Re: ZFS fans, rejoice – RAIDz expansion will be a thing soon

#175

Post by michaelmrose » Sat, Jun 19, 2021, 4:49 PM UTC

graton wrote: post #150

nullwarp wrote: post #123

Earlier quoted context omitted.

I've lost data in BTRFS setups each of the three times I've given it a try over the course of 6 or so years. Root drive just became unrecoverable. These are all single disk setups Meanwhile I've been running ZFS for close to the same time and have never lost anything. I get it's an anecdotal view point but that's a very hard reputation to rebuild for BTRFS.

Counter anecdotal view point, I've been using BTRFS over the last 4 years now on multiple drives and multiple systems. No data loss has occurred.

If it was so bad that nobody had positive anecdotes it would have to be nightmarish. It's like a car models reliability. A bad model new off the line is still liable to have years of trouble free usage just fewer on average. The difference between reliable and unreliable is the difference between 2 small negative probabilities over many iterations.

Re: ZFS fans, rejoice – RAIDz expansion will be a thing soon

#176

Post by nwmcsween » Sat, Jun 19, 2021, 5:18 PM UTC

magicalhippo wrote: post #165

nwmcsween wrote: post #107

I'm starting to get concerned about the ZFS issue list, there are a ton of gotchas hiding in using OpenZFS that will cause data loss: * Swap on ZVOL (data loss) * Hardlocking when removing ZIL (this has caused dataloss for us)

> Hardlocking when removing ZIL You cannot remove the ZIL, it's an integral part of ZFS. I assume you mean a SLOG[1]. That said, do you have an issue link? I'm curious. I know people have had some issues with importing pools with a bad or missing SLOG device, though most couldn't be reproduced. [1]: https://www.ixsystems.com/blog/o-slog-not-slog-best-configur...

Yes my mistake it was slog, this is easily reproducible w/ truenas, add cache, manually remove cache from cli, hardlock, there are multiple reports of this in the openzfs issue tracker

Re: ZFS fans, rejoice – RAIDz expansion will be a thing soon

#177

Post by nwmcsween » Sat, Jun 19, 2021, 5:21 PM UTC

michaelmrose wrote: post #154

nwmcsween wrote: post #107

I'm starting to get concerned about the ZFS issue list, there are a ton of gotchas hiding in using OpenZFS that will cause data loss: * Swap on ZVOL (data loss) * Hardlocking when removing ZIL (this has caused dataloss for us)

Can you please link me to the issue with swap on zvol causing data loss.

https://github.com/openzfs/zfs/issues/7734

Re: ZFS fans, rejoice – RAIDz expansion will be a thing soon

#178

Post by michaelmrose » Sat, Jun 19, 2021, 5:38 PM UTC

mixedCase wrote: post #57

d33lio wrote: post #2

I'll believe it when I see it, why anyone uses BTRFs (UnRaid or any other form of software raid that isn't ZFS) is still beyond me. At least when we're not talking SSD's ;) ZFS is incredible, curious to mess around with these new features!

I just put two 8TB drives into btrfs because it's a home server, I can't provision things up front. One day I may put a third 8TB drive and turn this RAID1 into RAID5. btrfs lets me do that, zfs doesn't, simple as. One day I may switch the whole thing to bcachefs, which I've donated and am looking forwards to. For the moment, btrfs will have to do. EDIT: downvoted by... the filesystem brigade?

I disagree with this statement on multiple fronts. On the first level of the onion RAID1 is for high reliability and BTRFS has historically low reliability but if you peel back that layer you are presenting the ability to transition from BTRFS RAID1 to RAID5 as an appealing feature of BTRFS vs ZFS and yet this just isn't so.

BTRFS has been promising usable RAID5 since 2009 when it was "heading for 1.0" and yet among the most recent developments not but 3 months ago was to add the following warning to btrfs-progs on creation or conversion.

"RAID5/6 support has known problems is strongly discouraged to be used besides testing or evaluation,"

Worse this feature was presented as usable around 2011/12 before being revealed to be unfixably data eating without substantial rewrites in 2016 and 5 years later remains so.

Your hardware might need to be replaced before you can avail yourself of the benefit you posit.

Meanwhile an approach that would actually work on both BTRFS and ZFS would be to add 2 drives to go from RAID1 to RAID10.

The last peel of onion is complaining about down votes. This invites more down votes. If I had to guess people down voted you because you presented a feature that has been a massive pain point for BTRFS as a proposed advantage.

Re: ZFS fans, rejoice – RAIDz expansion will be a thing soon

#179

Post by chousuke » Sat, Jun 19, 2021, 5:45 PM UTC

znpy wrote: post #153

chousuke wrote: post #143

Earlier quoted context omitted.

I'm not sure that talking about feature parity even makes sense. What do FreeBSD jails do that Linux can't? I know there's no single jail system call that you can call so you need to assemble the container "manually", but that's an implementation detail as far as I am concerned. On the other hand, true network namespaces (VNET on FreeBSD) have been usable on Linux for longer than on FreeBSD. It's a bit disingenuous t…

Ah, that bait got you. It's a smug comment that claims something but offers no example at all, not even an anecdotal one. If FreeBSD jails were so much better than Linux namespaces/containers etc we would be all using FreeBSD by now. The license is even more permissive. Containers have been out for ~ten years now and yet... Here we are.

I suspected it was bait, but FreeBSD jails being better than whatever Linux has is such a common meme that I'm asking anyway just in case someone actually knows enough about them to answer.

I don't know enough to say for sure, but my current impression is that Linux namespaces are actually more powerful than jails, but harder to use because the kernel doesn't provide a "simple" interface to create an isolated container, and userspace software implementing containers get to use the individual bits as they see fit.

Re: ZFS fans, rejoice – RAIDz expansion will be a thing soon

#180

Post by michaelmrose » Sat, Jun 19, 2021, 5:50 PM UTC

mixedCase wrote: post #75

nix23 wrote: post #67

Earlier quoted context omitted.

>RAID5 I wish you lots of fun with that on btrfs :) Edit: https://btrfs.wiki.kernel.org/index.php/Status RAID56 Unstable n/a write hole still exists > treated as if I'm storing business data or precious memories without backups, guess I'm just dumb No your not, but don't use unstable features in a filesystem

Well, that's the idea! This a low I/O media server where all the important stuff ( The whole point of me using RAID1 (and maybe later RAID5) is that if a disk goes bust, odds are I can still watch a movie from it until I can get another disk. What's more, if I ever fill the RAID1 and I don't feel like breaking the piggy bank for another disk, I can go JBOD as far as my usecase is concerned. But hey, if the orange web…

At 500 a disk you could fly a consultant out to demonstrate a high capacity storage solution consisting of many $200 disks who just happens to forget to bring the disks back!