In OpenZFS and Btrfs, everyone was just guessing

I have been more than satisfied with single-drive Btrfs with OpenSUSE so far, but nevertheless I can't wait for bcachefs to prove itself. On paper at least, bcachefs seems like the most attractive of the three.

For the bug in the commit linked up? I don't see how: the issue is a logic error causing a data race which depends on a whole bunch of factors a higher level language can't model directly.

The next step up for that is formal verification which is staggeringly time consuming (read: expensive) and hard to understand in the first place.

While I am firmly in the ZFS camp, my feeling is that there has indeed been a gradual slide in the disciplined development of ZFS.

After some initial bumps in brand new software, the Sun kernel group did a good job of avoiding corruption. Once Sun was absorbed by Oracle, OSS ZFS moved into illumos, who were overall quite good at doing the same (although they had less resources to play with). OpenZFS brought ZFS to the rest of the world (good), but I can't help but notice that the illumos devs are increasingly worried about pulling changes back from OpenZFS into illumos.

The increased popularity is for the best, but the change rate has increased, with all that entails.

I don't know about BTRFS, but with ZFS there's a lot of bad information and assumptions floating around. It's one of my favorite topics to ask AI about because it shows the obvious deficiencies in AI as it regurgitates all the bad info. So when you go to the issue tracker, it seems plausible to me that you're going to find issues where the devs can't explain what's happening because the user(s) might be doing something crazy.

With the exception of a couple people like Jim Salters, I don't take any advice I see about ZFS unless I know the person is a dev or has commits on the project. And most of the time I take the advice I've seen 'ryao' post which is, paraphrased, "use the defaults unless you know what you're tuning". That should go without saying, but there are a lot of armchair "geniuses" posting online about ZFS and they are just guessing.

And, how to you get any new developers if the expectation is an immediate, complete understanding of the system? That just isn't realistic and, in most cases, the dev that knows a little bit probably knows exponentially more than an average user. Interacting on the bug tracker and debugging things they don't completely understand is going to improve their understanding. There's value there.

Things have gotten more complicated too. We aren't using spinning disks without caches any more, right? I can understand a disk from 20 years ago. They're relatively simple. I couldn't tell you the first thing about SSDs or NVMe stuff.

There are going to be bugs. Just be glad the devs acknowledge them and fix them. Personally I still rank ZFS as the least likely to lose my data and I'll keep using it. I put roughly zero value in a forum full of people complaining.

You're at odds with the desire (necessity, actually) to have precise low-level control, not just of the in-memory layout of the data structures, but also of the performance characteristics of your code (e.g. no unnecessary pointer-chasing). Higher-level languages tend to make it easier to pile up abstractions; we want the orthogonal property of making it harder to shoot yourself in the foot.

You can look at ZFS for inspiration. It is more complex than say ext4, but it's less complex than the full stack of mdadm+LVM+ext4 - the latter piles up the abstractions, where ZFS is able to "reach around" and e.g. directly track free physical blocks across multiple devices. So when you need to resilver the pool, you don't need to copy the unused blocks, and reduce the load on the pool (and thus the chance of double failure).

Are there programming languages that have such properties? At the risk of perpetuating the meme, I'd say Rust (with no_std) fits that description. I can't tell how would it help in this specific instance (I'm very far from an expert in FS implementation), but it does tend to prevent data races in general.