Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Bcachefs Goes to "Externally Maintained"

lwn.net

Post reply on HN
Print view View original

301–310 of 400 posts

Re: Bcachefs Goes to "Externally Maintained"

#301

Post by m-p-3 »

dmm wrote: post #66
ofrzeta wrote: post #28

Earlier quoted context omitted.

Suse Linux Enterprise still uses Btrfs as the Root-FS, so it can't be that bad, right? What is Chris Mason actually doing these days? I did some googling and only found out that he was working on a tool called "rsched".

btrfs is fine for single disks or mirrors. In my experience, the main advantages of zfs over btrfs is that ZFS has production ready raid5/6 like parity modes and has much better performance for small sync writes, which are common for databases and hosting VM images.

Context: I mostly dealt with RAID1 in a home NAS setup

A ZFS pool will remain available even in degraded mode, and correct me if I'm wrong but with BTRFS you mount the array through one of the volume that is part of the array and not the array itself.. so if that specific mounted volume happens to go down, the array becomes unavailable unmounted until you remount another available volume that is part of the array which isn't great for availability.

I thought about mitigating that by making an mdadm RAID1 formatted with BTRFS and mount the virtual volume instwad, but then you lose the ability to prevent bit rot, since BTRFS lose that visibility if it doesn't manage the array natively.

Re: Bcachefs Goes to "Externally Maintained"

#302

Post by p_ing »

koverstreet wrote: post #246
mort96 wrote: post #224

Earlier quoted context omitted.

It's so sad to see an excellent engineer such as yourself, building what seems like an excellent filesystem that has the potential to be better than everything else available for Linux for many use cases, completely fail to achieve your goals because you lack the people skills to navigate working as a part of a team under a technical leader. Every comment and e-mail I've seen from you has demonstrated an impressive l…

I get a ton of comments like this. Pointing the finger at the skills I lack and my inability, while ignoring the wider picture, of the kernel burning out maintainers and not doing well on filesystems. It's wearying.

When everyone else is the problem, you're the problem.

Consider that when working in teams.

Re: Bcachefs Goes to "Externally Maintained"

#303

Post by koverstreet »

mort96 wrote: post #298
koverstreet wrote: post #286

Earlier quoted context omitted.

That's because it is my project, and my responsibility. I can't be bowing to the demands of any one person; I have to balance the wants and needs of everyone and prioritize shipping something that works above all else. Repeatedly we've seen that those priorities are not shared, unfortunately. Arguments are just as heated as they ever were, but now instead of arguing over the actual issues - does this work, are we doi…

> I can't be bowing to the demands of any one person This right here is the core of the issue. When you're working as a part of a larger organizational structure, you have to bow down to your boss. When your software is a part of the kernel, it's not your project anymore; it's just one part of Linus's project. You're a contributor, not a leader. Just like I would not control Bcachefs's development process even if I c…

Linus isn't my boss, though.

Am I paid by him or the Linux foundation? No.

Has he ever contributed to bcachefs in any way, is he in any way responsible for making sure that it works properly? No.

The only sense in which he has authority is that he can decide whether or not to pull it into his tree, but that's a two way relationship.

Re: Bcachefs Goes to "Externally Maintained"

#304

Post by quotemstr »

nullc wrote: post #282
quotemstr wrote: post #274

Earlier quoted context omitted.

You're staking out quite the postmodernist position there. All models are wrong, so who's to say that Alice's data corruption is worse than Bob's man page typo? The important thing is we stick to process with a proven track record, right? I don't buy it. Object level considerations do matter. Alice's bug really is worse than Bob's. That "proven track record" shouldn't apply to Alice, and insisting that it does for th…

> Object level considerations do matter. They do. And Kent expressed them and the linux kernel maintainers are amply qualified to hear out and make a call. I don't see a reason to think they were indifferent to the facts, they just weren't convinced by them. If they were they could have just said, "okay we think that this does qualify as a bugfix". My understanding is the change in dispute wasn't over fixing the corr…

Thanks for the thoughtful reply.

> I don't see a reason to think they were indifferent to the facts

I don't think the Linux people thought of themselves as indifferent to facts. Nor do I think they were, not at first. Most people imagine themselves as fair-minded truth-seekers. When stakes are low, they usually act like it. It's only under pressure that people reveal whether they're more committed to PR or progress.

The shitty thing about this situation is that as the dispute escalated, the technical merits of change faded from relevance. (Linus even pulled the corruption repair work in the end!) The argument transformed into a dispute over power, pride, and personalities. Linus's commitment to technical excellence was tested. It failed. Consequently, Linux will lack a cutting-edge filesystem.

I don't even object to Linus being BDFL of Linux. Somebody has to make decisions. I think Linus was wrong to reject the corruption fix patch, but he could plausibly have been right. He had an opportunity to explain his patch rejection in such a way that Overstreet would have understood it as final but also felt heard and valued. Overstreet would have been upset, and justifiably so, but by the next merge window both sides would have cooled down and progress would have resumed.

It's when Linus banned Overstreet and bcachefs from the project that he departed irrecoverably from defensibility. Linus might think he's punishing Overstreet for his intransigence by blocking his work, but Linus is actually taking his frustration out on every Linux user instead. Overstreet's ban is rooted in primate power psychology, not technical trade-offs, and it makes everyone lose.

Technical leaders who ostracize brilliant but difficult people forever cap the amount of progress we can make in the fight against the limits of nature. They're neglecting their responsibilities as leaders to harness difficult people. It's not an easy job, but being a leader shouldn't be.

Linus took the easy way out and banned the brilliant troublemaker. He should be ashamed.

> the risk is not measurable on a case by case basis

It often is. That's why when I'm on the Linus side of a case like this, I try to avoid saying "no" and instead say "yes, if". Sometimes my counterparty pulls out an "if" that convinces me.

Re: Bcachefs Goes to "Externally Maintained"

#305

Post by mort96 »

koverstreet wrote: post #303
mort96 wrote: post #298

Earlier quoted context omitted.

> I can't be bowing to the demands of any one person This right here is the core of the issue. When you're working as a part of a larger organizational structure, you have to bow down to your boss. When your software is a part of the kernel, it's not your project anymore; it's just one part of Linus's project. You're a contributor, not a leader. Just like I would not control Bcachefs's development process even if I c…

Linus isn't my boss, though. Am I paid by him or the Linux foundation? No. Has he ever contributed to bcachefs in any way, is he in any way responsible for making sure that it works properly? No. The only sense in which he has authority is that he can decide whether or not to pull it into his tree, but that's a two way relationship.

I have said my piece. It's not me you have to convince.

Re: Bcachefs Goes to "Externally Maintained"

#306

Post by motorest »

nullc wrote: post #287
motorest wrote: post #272

Earlier quoted context omitted.

> Which also, at times, means appeasing people even when you are confident that they are wrong because you need their cooperation in the future. Being unwilling to follow basic QA processes in preparation of a release candidate, and then doubling down by attacking the release engineer with claims the QA process doesn't apply to you because you know better, is something that is far more serious than lacking basic soft…

> It's a fireable offense in most companies. In a company there are other employees who have your success as part of their job function. People to train you, to talk you down off a ledge, people to step in and guard you against misunderstanding or criticisms. People to advocate for you or send you home before a dispute crosses a point of no return. You're also paid to be there, to put up with the companies BS, .. the…

> In a company there are other employees who have your success as part of their job function.

Yes, and they enforce basic relase processes to ensure you don't break releases by skipping QA processes or introducing untested and unverified features in release candidates.

And you sure as hell don't have primadonna developers stay in the payroll for long if they start throwing tantrums and personal attacks towards fellow engineers when they are asked to follow the release process or are called out for trying to sneak untested changes in mission-critical components.

Re: Bcachefs Goes to "Externally Maintained"

#307

Post by koverstreet »

whatagreatboy wrote: post #288
koverstreet wrote: post #101

Earlier quoted context omitted.

No, the problem wasn't following the rules. The patch that kicked off the current conflict was the 'journal_rewind' patch; we recently (6.15) had the worst bug in the entire history upstream - it was taking out entire subvolumes. The third report got me a metadata dump with everything I needed to debug the issue, thank god, and now we have a great deal of hardening to ensure a bug like this can never happen again. Su…

Was there any attempt at making rules for experimental features looser than other filesystems? That seems to be the biggest bottleneck here.

That does seem to be one of the big disconnects, yes.

In the past I've argued that I do need a relatively free hand and to be able to move quickly, and explained my reasoning: we've been at the stage of stabilization where the userbase is fairly big, and when someone reports a bug we really need to work with them and fix it in a timely manner in order to keep them testing and reporting bugs. When someone learns the system well enough to report a bug, that's an investment of time and effort on their part, and we don't want to lose that by having them get frustrated and leave.

IOW: we need to prioritize working with the user community, not just the developer community.

All that's been ignored though, and the other kernel maintainers seem to just want to ratchet down harder and harder and harder on strictness.

At this point, we're past the bulk of stabilization, and I've seen (to my surprise) that I've actually been stricter with what I consider a critical fix than other subsystems.

So this isn't even about needing special rules for experimental; this is just about having sane and consistent rules, at all.

Re: Bcachefs Goes to "Externally Maintained"

#308

Post by magicalhippo »

koverstreet wrote: post #266
p_l wrote: post #229

Earlier quoted context omitted.

The lack of traditional 'fsck' is because its operation would be exact same as normal driver operation. The most extreme case involves a very obscure option that lets you explicitly rewind transactions to one you specify, which I've seen used to recover a broken driver upgrade that led to filesystem corruption in ways that most FSCK just barf on, including XFS' For low-level meddling and recovery, there's a filesyste…

Rewinding transactions is cool. Bcachefs has that too :) What happens on ZFS if you lose all your alloc info? Or are there other single points of failure besides the ublock in the on disk format?

> What happens on ZFS if you lose all your alloc info?

According to this[1] old issue, it hasn't happened frequently enough to prioritize implementing a rebuild option, however one should be able to import the pool read-only and zfs send it to a different pool.

As far as I can tell that's status quo. I agree it is something that should be implemented at some point.

That said, certain other spacemap errors might be recoverable[2].

[1]: https://github.com/openzfs/zfs/issues/3210

[2]: https://github.com/openzfs/zfs/issues/13483#issuecomment-120...

Re: Bcachefs Goes to "Externally Maintained"

#309

Post by koverstreet »

motorest wrote: post #258
koverstreet wrote: post #248

Earlier quoted context omitted.

No, I'm sorry but you're simply wrong. bcachefs has a ton of QA, both automated testing and a lot of testers that run my latest and I work with on a daily basis. The patch was well tested; it was for codepaths that we have good regression tests for, it was algorithmically simple, and it worked perfectly to recover a filesystem from the original bug report, and it performed flawlessly again not long after. I've explai…

> No, I'm sorry but you're simply wrong. It sounds like you have a hard time coping with reality. https://www.phoronix.com/news/Linux-616-Bcachefs-Late-Featur... I repeat: it sounds an awful lot like you are trying to gaslight this thread. Not cool. When this fact was again explicitly pointed out to you by Linus himself, you even tried to bullshit Linus and try to move the goalpost with absurd claims about how someho…

The changes weren't untested or unreviewed, and they've performed flawlessly on quite a few occasions since then.

Sorry, the only person gaslighting here is you.

Re: Bcachefs Goes to "Externally Maintained"

#310

Post by koverstreet »

ranger_danger wrote: post #291
charcircuit wrote: post #128

Earlier quoted context omitted.

Your analogy fails to account that after "Friday" bug fixes are still allowed. A file system losing your files sounds like a bug to me. Edit since you expanded your post: >The most interesting to me was LWN, about the debian tools, where an actual psychologist got involved. To me the comment was patronizing implying it was purely due to bad communication from Kent's end and shows how immature people are with running…

[flagged]

You think I have an alt with higher karma than my actual account? :)
Post reply on HN