Live data from Hacker News

Bcachefs Goes to "Externally Maintained"

lwn.net

291–300 of 400 posts

Re: Bcachefs Goes to "Externally Maintained"

#291

Earlier quoted context omitted.

I think this attitude is exactly why this happened. I would have done the same thing. Do you argue with your school teachers that your book report shouldn't be due on Friday because it's not perfect yet? I read several of your response threads across different websites. The most interesting to me was LWN, about the debian tools, where an actual psychologist got involved. All the discussions seem to show the same issu…

Your analogy fails to account that after "Friday" bug fixes are still allowed. A file system losing your files sounds like a bug to me. Edit since you expanded your post: >The most interesting to me was LWN, about the debian tools, where an actual psychologist got involved. To me the comment was patronizing implying it was purely due to bad communication from Kent's end and shows how immature people are with running…

[flagged]

Re: Bcachefs Goes to "Externally Maintained"

#292
post #275

Earlier quoted context omitted.

You'd think that, wouldn't you? But there are enough moving parts in the IO stack below the filesystem that we do see bit errors. I don't have enough data to do correlations and tell you likely causes, but they do happen. I think SSDs are generally worse than spinning rust (especially enterprise grade SCSI kit), the hard drive vendors have been at this a lot longer and SSDs are massively more complicated. From the co…

One thing to keep in mind is that correction always comes as some expense of detection. Generally a code that can always detect N errors can only always correct N/2 errors. So you detect an errored block, you correct up to N/2 errors. The block now passes but if the block actually had N errors, your correction will be incorrect and you now have silent corruption. The solution to this is just to have an excess of erro…

bcachefs checksums (and compresses) at extent granularity, not block; encoded extents (checksummed/compressed) are limited to 128k by default.

This is a really good tradeoff in practice; the vast majority of applications are doing buffered IO, not small block O_DIRECT reads - that really only comes up in benchmarks :)

And it gets us better compression ratios and better metadata overhead.

We also have quite a bit of flexibility to add something bigger to the extent for FEC, if we need to - we're not limited to a 32/64 bit checksum.

Re: Bcachefs Goes to "Externally Maintained"

#293
post #59
post #9

Is it just me or does Kent seem self-destructively glued to his own idea of how kernel development should work? I don’t doubt that people on all sides have made mis-steps, but from the outside it mostly just seems like Kent doesn’t want to play by the rules (despite having been given years of patience).

I think Kent is in the wrong here, but it really doesn't help that the kernel people from Linus on down are seemingly unable to explain the problem, and instead resort to playground insults. Apart from being unprofessional and making for a hostile work environment, it doesn't really communicate why Kent's actions are problematic, so I've some sympathy for his not believing that they are.

> unable to explain the problem

unfortunately that's either due to lack of investigation by yourself or a bit dishonest.

Re: Bcachefs Goes to "Externally Maintained"

#294

Earlier quoted context omitted.

Parity support in multi-disk arrays is older than I am, it's a fairly standard feature. btrfs doesn't support this without data loss risks after 17 years of development.

If you're not interested in a multi-disk storage system that doesn't have (stable, non-experimental) parity modes, that's a valid personal preference but not at all a justification for the position that the rest of the features cannot be stable and that the project as a whole cannot be taken seriously by anyone.

Is that what I said?

Re: Bcachefs Goes to "Externally Maintained"

#295
post #127
post #12

The sad part, that despite the years of the development BTRS never reached the parity with ZFS. And yesterday's news "Josef Bacik who is a long-time Btrfs developer and active co-maintainer alongside David Sterba is leaving Meta. Additionally, he's also stepping back from Linux kernel development as his primary job." see https://www.phoronix.com/news/Josef-Bacik-Leaves-Meta There is no 'modern' ZFS-like fs in Linux n…

There's literally ZFS-on-linux and it works great. And yes, I will once again say Linus is completely wrong about ZFS and the multiple times he's spoken about it, it's abundantly clear he's never used it or bothered to spend any time researching its features and functionality. https://zfsonlinux.org/

I've recently started using OpenZFS after all these years, and after weighting all the pros and cons of BTRFS, mdadm, etc, ZFS is clearly on top for availability and resiliency.

Hopefull we can get to a point where Linux has a native, and first-class modern alternative to ZFS with BcacheFS.

Re: Bcachefs Goes to "Externally Maintained"

#296
post #289

Earlier quoted context omitted.

that would be bcachefs :) It's an entirely clean slate design, and I spent years taking my time on the core planning out the design; it's as close to perfect as I can make it. The only things I can think of that I would change or add given unlimited time and budget: - It should be written in Rust, and even better a Rust + dependent types (which I suspect could be done with proc macros) for formal verification. And ca…

Thanks for bcachefs and all the hard work you’ve put in it. It’s truly appreciated and hope you can continue to march on and not give up on the in-kernel code, even if it means bowing to Linus. On a different note, have you heard about prolly trees and structural sharing? It’s a newer data structure that allows for very cheap structural sharing and I was wondering if it would be possible to build an FS on top of it t…

I have not seen those...

Re: Bcachefs Goes to "Externally Maintained"

#297

Earlier quoted context omitted.

> But there's a ton of room for improvement beyond what ZFS did. Say more? I can't say I've really thought that much about filesystems and I'm curious in what direction you think they could be taken if time and budget weren't an issue.

that would be bcachefs :) It's an entirely clean slate design, and I spent years taking my time on the core planning out the design; it's as close to perfect as I can make it. The only things I can think of that I would change or add given unlimited time and budget: - It should be written in Rust, and even better a Rust + dependent types (which I suspect could be done with proc macros) for formal verification. And ca…

I'm saddened by this turn of event, but I hope this won't deter you from working on bcachefs on your own term and eventually see a reconciliation into the kernel at one point.

Thank you for your hard work.

Re: Bcachefs Goes to "Externally Maintained"

#298
post #255

Earlier quoted context omitted.

You get a ton of comments like this because it's true. There are real problems in the kernel, I've seen how hostile it can be to people who are just trying to do the right thing and upstream their changes etc. But your case isn't that. Your behavior would get you in trouble at any job where you have to follow rules set by other people. Your refusal to treat your part of the kernel as anything other than your personal…

That's because it is my project, and my responsibility. I can't be bowing to the demands of any one person; I have to balance the wants and needs of everyone and prioritize shipping something that works above all else. Repeatedly we've seen that those priorities are not shared, unfortunately. Arguments are just as heated as they ever were, but now instead of arguing over the actual issues - does this work, are we doi…

> I can't be bowing to the demands of any one person

This right here is the core of the issue. When you're working as a part of a larger organizational structure, you have to bow down to your boss. When your software is a part of the kernel, it's not your project anymore; it's just one part of Linus's project. You're a contributor, not a leader. Just like I would not control Bcachefs's development process even if I contributed some small but important part to it, you do not control Linux's development process even though you contributed some small but important part to it.

Your core mission is evidently not shipping a reliable trustworthy filesystem. You say that, but your actions speak louder than your words. You know just as well as I do that a filesystem being in-tree rather than out-of-tree makes it significantly more reliable and trustworthy, which is why you chose to get Bcachefs merged into the kernel in the first place. Instead of working within the well-defined boundaries that's necessary to keep Bcachefs in the kernel, you've repeatedly pushed against those boundaries, belittled fellow maintainers, and in general worked hard to make yourself a persona non grata within the kernel community. The predictable outcome is that continued development of Bcachefs will have to happen out-of-tree, and your users won't gain the major reliability and trustworthiness benefits of using an in-tree filesystem. People will warn against using Bcachefs as their root filesystem, since every kernel upgrade will now carry some risk that DKMS or whatever mechanism is used to install the out-of-tree Bcachefs kernel module doesn't work with the new kernel.

And, to be honest, it doesn't matter whether or not you're "right" or "wrong" here. Maybe you're completely correct about absolutely everything and Linus, Greg, Ted, Miguel, Sasha, Josef, and everyone else involved are stupid and don't understand what it takes to develop reliable software. So what? They're your colleagues, some of them are your bosses. Everyone on Hacker News could take your side here and think you've been mistreated, it doesn't help. You'd still be thrown out of the kernel. You'd still be failing your users by not maintaining a good enough relationship with your colleagues and bosses to stay in-tree. You could be completely right on every technical matter and it does not matter.

If you play your cards right, you could maybe end up in a situation where you run the Bcachefs project entirely out-of-tree, with yourself as the supreme leader who doesn't bow down to the demands of anyone, with your own development and release process; and then someone else takes responsibility for pushing your code into the upstream kernel, following Linus's rules. They would dissect your releases and backport bug fixes while leaving out important features, in accordance with Linus's rules. Time will tell if you can find anyone to do that. And time will tell if you posess the humility necessary to let someone else ultimately control the experience most of your users will have.

Re: Bcachefs Goes to "Externally Maintained"

#299
post #279

Earlier quoted context omitted.

Again: the elephant in the room is that a lot of bcachefs users are using it explicitly because they have lost a lot of data on btrfs, and they've found it to be more trustworthy. This puts us all in a shitty situation. I want the experimental label to come off at the right time - when every critical bug is fixed and it's as trustworthy as I can reasonably make it, when I know according to the data I have that everyo…

>users are using it explicitly because they have lost a lot of data on btrfs PLEASE, honestly, EDUCATE THESE USERS. This is still marked experimental for numerous reasons regardless of the 'planned work for 6.18'. Users who can't suffer any data loss and are repeating their mistake of using btrfs shouldn't be using a none default/standard/hardened filesystem period.

No, really. People aren't losing data on bcachefs. We still have minor hiccups that do affect usability, and I put a lot of effort into educating users about where we're at and what to expect.

In the past I've often told people who wanted to migrate off of btrfs "check back in six months", but I'm not now because 6.16 is looking amazingly solid; all the data I have says that your data really is safer on bcachefs than btrfs.

I'm not advocating for people to jump from ext4/xfs/zfs, that needs more time.

Re: Bcachefs Goes to "Externally Maintained"

#300
post #241
post #213

Earlier quoted context omitted.

Oracle does not have copyright on OpenZFS code - only on the version in Solaris. The code in OpenZFS and Solaris has diverged after Oracle closed OpenSolaris.

> The code in OpenZFS and Solaris has diverged after Oracle closed OpenSolaris. Diverged . Not rewritten entirely.

Sure, but Oracle cannot retroactively relicense the code already published before then. The cat's already out of the bag, and as long as the code from before the fork is used according to the original license, it's legal.
Post reply on HN