Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Bcachefs Goes to "Externally Maintained"

lwn.net

Post reply on HN
Print view View original

281–290 of 400 posts

Re: Bcachefs Goes to "Externally Maintained"

#281

Post by qalmakka »

icar wrote: post #187
qalmakka wrote: post #141

The whole situation is moronic at best. Linux needs a decent modern filesystem in tree. ZFS would easily be it, but unfortunately Sun decided back in the '00s to fuck Linux because they wanted to push Solaris instead. Little they knew ZFS ended up being FreeBSD top feature for years. Btrfs is constantly eating people data, it's a bad joke nowadays. Right now on Linux you're basically forced to constantly deal with ou…

I know it's not the same, but close enough for me: lvm + xfs works wonders and it's rock solid.

Thin provisioning with lvm is afaik problematic, because the file systems aren't aware of it and if you run out of space you're screwed

Re: Bcachefs Goes to "Externally Maintained"

#282

Post by nullc »

quotemstr wrote: post #274
nullc wrote: post #256

Earlier quoted context omitted.

Linux is not correct. Linux has never been correct. Linux will never be correct. An incorrect belief that it is correct can only make it less correct. You must know this when it comes to your own work. Why isn't bcachefs written in augmented rust with dependent types and formal correctness proofs for every line of code? How could there ever be a data losing bug if you had a formal proof that the file system could nev…

You're staking out quite the postmodernist position there. All models are wrong, so who's to say that Alice's data corruption is worse than Bob's man page typo? The important thing is we stick to process with a proven track record, right? I don't buy it. Object level considerations do matter. Alice's bug really is worse than Bob's. That "proven track record" shouldn't apply to Alice, and insisting that it does for th…

> Object level considerations do matter.

They do. And Kent expressed them and the linux kernel maintainers are amply qualified to hear out and make a call. I don't see a reason to think they were indifferent to the facts, they just weren't convinced by them. If they were they could have just said, "okay we think that this does qualify as a bugfix".

My understanding is the change in dispute wasn't over fixing the corruption introducing bug, but rather adding automated repair for cases where the corruption had already happened. I could easy see taking a position of "sad for people who are already corrupt, they can get their work around out of tree for now" (or heck, even forever depending on the scale of the impact).

Anyone who has been around for a while has seen their share of 'ate the horse to catch the spider to catch the fly to...' dance, of course the patch author is convinced that their repair is correct. They're almost always convinced of that or they don't submit it, so that carries little information. Because of this there is a strong preference for obviously minimal code in any kind of fix. Minimizing user suffering is important, but we also know every line of code comes with risk. The fact that the risk is not measurable on a case by case basis doesn't make it any less real.

Re: Bcachefs Goes to "Externally Maintained"

#283

Post by rob_c »

quotemstr wrote: post #168
nirava wrote: post #104

Earlier quoted context omitted.

To list down the current state of things: 1. Regardless of whether correct or not, it's Linus that decides what's a feature and what's not in Linux. Like he has for the last however many decades. Repair code is a feature if Linus says it is a feature. 2. Being correct comes second to being agreeable in human-human interactions. For example, dunking on x file system does not work as a defense when the person opposite…

> Being correct comes second to being agreeable in human-human interactions Prioritizing agreeableness above correctness is the reason the space shuttle Challenger blew up. The bcachefs fracas is interesting and important because it's like a stain making some damn germ's organelles visible: it highlights a psychological division in tech and humanity in general between people who prioritize 1) deferring to authority,…

> Prioritizing agreeableness above correctness is the reason the space shuttle Challenger blew up.

Oh dear lord no. That is not even what _any_ of the actual investigations suggested.

woke agreeableness is bad but it wasn't getting along at the water-cooler that lead to challenger.

Re: Bcachefs Goes to "Externally Maintained"

#284

Post by dmm »

riku_iki wrote: post #89
dmm wrote: post #66

Earlier quoted context omitted.

btrfs is fine for single disks or mirrors. In my experience, the main advantages of zfs over btrfs is that ZFS has production ready raid5/6 like parity modes and has much better performance for small sync writes, which are common for databases and hosting VM images.

> has much better performance for small sync writes I spent some time researching this topic, and in all benchmarks I've seen and my personal tests btrfs is faster or much faster: https://www.reddit.com/r/zfs/comments/1i3yjpt/very_poor_perf...

Thanks for sharing! I just setup a fs benchmark system and I'll run your fio command so we can compare results. I have a question about your fio args though. I think "--ioengine=sync" and "--iodepth=16" are incompatible, in the sense that iodepth will only be 1.

"Note that increasing iodepth beyond 1 will not affect synchronous ioengines"[1]

Is there a reason you used that ioengine as opposed to, for example, "libaio" with a "--direct=1" flag?

[1] https://fio.readthedocs.io/en/latest/fio_doc.html#cmdoption-...

Re: Bcachefs Goes to "Externally Maintained"

#285

Post by rob_c »

masklinn wrote: post #179
shmerl wrote: post #135

Some collaboration failing there, no technical reasons for it. I hope they'll sort this nonsense out and it will go back to normal upstream maintenance.

It won’t. Kent is in this very post (as well as TFA) showing that he’s not learned anything

Shame, at the very least I was hoping this would lead to approaching someone to work as a go between.

(I find it difficult to believe nobody tried to step up to that position either)

Re: Bcachefs Goes to "Externally Maintained"

#286

Post by koverstreet »

mort96 wrote: post #255
koverstreet wrote: post #246

Earlier quoted context omitted.

I get a ton of comments like this. Pointing the finger at the skills I lack and my inability, while ignoring the wider picture, of the kernel burning out maintainers and not doing well on filesystems. It's wearying.

You get a ton of comments like this because it's true. There are real problems in the kernel, I've seen how hostile it can be to people who are just trying to do the right thing and upstream their changes etc. But your case isn't that. Your behavior would get you in trouble at any job where you have to follow rules set by other people. Your refusal to treat your part of the kernel as anything other than your personal…

That's because it is my project, and my responsibility.

I can't be bowing to the demands of any one person; I have to balance the wants and needs of everyone and prioritize shipping something that works above all else.

Repeatedly we've seen that those priorities are not shared, unfortunately.

Arguments are just as heated as they ever were, but now instead of arguing over the actual issues - does this work, are we doing this right - people jump to arguing over language and conduct and demanding apologies or calling for people to be expelled.

But my core mission is just shipping a reliable trustworthy filesystem, and that's what I'm going to stick to.

Re: Bcachefs Goes to "Externally Maintained"

#287

Post by nullc »

motorest wrote: post #272
nullc wrote: post #240

Earlier quoted context omitted.

> minimum avoid pissing everyone else off Which also, at times, means appeasing people even when you are confident that they are wrong because you need their cooperation in the future. In a large complicated system, being able to work together is often more important to the system's reliability, performance, etc. than being as right as possible. Plus even when you're confident you are in the right you might still be…

> Which also, at times, means appeasing people even when you are confident that they are wrong because you need their cooperation in the future. Being unwilling to follow basic QA processes in preparation of a release candidate, and then doubling down by attacking the release engineer with claims the QA process doesn't apply to you because you know better, is something that is far more serious than lacking basic soft…

> It's a fireable offense in most companies.

In a company there are other employees who have your success as part of their job function. People to train you, to talk you down off a ledge, people to step in and guard you against misunderstanding or criticisms. People to advocate for you or send you home before a dispute crosses a point of no return. You're also paid to be there, to put up with the companies BS, .. the project isn't yours, it's not usually your reputation that's hurt when the company wants to make a decision you don't agree with and it goes poorly.

The context is so different, I don't think it's really comparable.

Re: Bcachefs Goes to "Externally Maintained"

#288

Post by whatagreatboy »

koverstreet wrote: post #101
tarruda wrote: post #46

Since the existing bcachefs driver will not be removed, and the problem is the bcachefs developer not following the rules, I wonder if someone else could take on the role of pulling bcachefs changes into the mainline, while also following the merge window rules.

No, the problem wasn't following the rules. The patch that kicked off the current conflict was the 'journal_rewind' patch; we recently (6.15) had the worst bug in the entire history upstream - it was taking out entire subvolumes. The third report got me a metadata dump with everything I needed to debug the issue, thank god, and now we have a great deal of hardening to ensure a bug like this can never happen again. Su…

Was there any attempt at making rules for experimental features looser than other filesystems? That seems to be the biggest bottleneck here.

Re: Bcachefs Goes to "Externally Maintained"

#289

Post by lifty »

koverstreet wrote: post #159
TheAceOfHearts wrote: post #147

Earlier quoted context omitted.

> But there's a ton of room for improvement beyond what ZFS did. Say more? I can't say I've really thought that much about filesystems and I'm curious in what direction you think they could be taken if time and budget weren't an issue.

that would be bcachefs :) It's an entirely clean slate design, and I spent years taking my time on the core planning out the design; it's as close to perfect as I can make it. The only things I can think of that I would change or add given unlimited time and budget: - It should be written in Rust, and even better a Rust + dependent types (which I suspect could be done with proc macros) for formal verification. And ca…

Thanks for bcachefs and all the hard work you’ve put in it. It’s truly appreciated and hope you can continue to march on and not give up on the in-kernel code, even if it means bowing to Linus.

On a different note, have you heard about prolly trees and structural sharing? It’s a newer data structure that allows for very cheap structural sharing and I was wondering if it would be possible to build an FS on top of it to have a truly distributed fs that can sync across machines.

Re: Bcachefs Goes to "Externally Maintained"

#290

Post by riku_iki »

dmm wrote: post #284
riku_iki wrote: post #89

Earlier quoted context omitted.

> has much better performance for small sync writes I spent some time researching this topic, and in all benchmarks I've seen and my personal tests btrfs is faster or much faster: https://www.reddit.com/r/zfs/comments/1i3yjpt/very_poor_perf...

Thanks for sharing! I just setup a fs benchmark system and I'll run your fio command so we can compare results. I have a question about your fio args though. I think "--ioengine=sync" and "--iodepth=16" are incompatible, in the sense that iodepth will only be 1. "Note that increasing iodepth beyond 1 will not affect synchronous ioengines"[1] Is there a reason you used that ioengine as opposed to, for example, "libaio…

Intuition is that majority of software uses standard sync FS api..
Post reply on HN