Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Bcachefs Goes to "Externally Maintained"

lwn.net

Post reply on HN
Print view View original

391–400 of 400 posts

Re: Bcachefs Goes to "Externally Maintained"

#391

Post by b112 »

koverstreet wrote: post #309
motorest wrote: post #258

Earlier quoted context omitted.

> No, I'm sorry but you're simply wrong. It sounds like you have a hard time coping with reality. https://www.phoronix.com/news/Linux-616-Bcachefs-Late-Featur... I repeat: it sounds an awful lot like you are trying to gaslight this thread. Not cool. When this fact was again explicitly pointed out to you by Linus himself, you even tried to bullshit Linus and try to move the goalpost with absurd claims about how someho…

The changes weren't untested or unreviewed, and they've performed flawlessly on quite a few occasions since then. Sorry, the only person gaslighting here is you.

Holy cow. You really deserved the kick. Fixes the RC in question aren't for bugfixes in general, but bugfixes for existing patches that in that RC.

It doesn't matter if you have bugs in the code still. At all. Having bugs in the code is not relevant. You are 100%, completely, inexcusably wrong if you were providing a bugfix that wasn't fixing something specifically changed in the RC.

Untested and unreviewed? Your own testing process and review process does not meet that. "Since then" is entirely irrelevant. Saying "I've proven it works" means you have no concept of stability in a release process.

I sincerely hope your code gets the boot from the kernel, because you clearly will never be able to maintain it there. You just don't listen. To anyone.

Your personal definitions of the above terms are irrelevant. How you think the kernel should be maintained, is meaningless. Whether you think your code is ready is an insane way to validate that you're "super special and get to have your way".

Personally, I'd never hire you from this alone.

Re: Bcachefs Goes to "Externally Maintained"

#392

Post by procaryote »

aaronmdjones wrote: post #390
procaryote wrote: post #389

Earlier quoted context omitted.

Won't silent corruption on the raid level be detected by the integrity checks in btrfs? It won't be able to automatically repair it, but it should give errors at least, right?

Yeah, that would be the "error detection at a higher level" (than MD) part. It'd still be on you to pull one drive at a time from the array until the errors go away (then you know which drive has the corrupted block in that stripe, and can remove the mdadm metadata from it and then re-add it to the array so that the kernel forces a clean resync, reconstructing the good block from the parity). Doing the "repair" actio…

Sure, you might have some work on corruption but it won't be silent and it would be recoverable without data loss

I'd personally replace rather than re-add a drive with corruption but perhaps I'm overly paranoid

Re: Bcachefs Goes to "Externally Maintained"

#393

Post by aaronmdjones »

procaryote wrote: post #392
aaronmdjones wrote: post #390

Earlier quoted context omitted.

Yeah, that would be the "error detection at a higher level" (than MD) part. It'd still be on you to pull one drive at a time from the array until the errors go away (then you know which drive has the corrupted block in that stripe, and can remove the mdadm metadata from it and then re-add it to the array so that the kernel forces a clean resync, reconstructing the good block from the parity). Doing the "repair" actio…

Sure, you might have some work on corruption but it won't be silent and it would be recoverable without data loss I'd personally replace rather than re-add a drive with corruption but perhaps I'm overly paranoid

If it were only one data block in one stripe I'd be confident re-adding the same drive (and have done so); this is overwhelmingly likely to be a transient error (e.g. bit rot on the drive or a RAM bit flip while writing; either in the drive itself or the machine's main memory) that won't recur.

The MD "check" action can confirm this (it will iterate every stripe and report all parity/data mismatches, so if it only reports one ...) and some distributions ship a cronjob that automatically does this on a monthly basis.

If it were a corrupt parity block in a stripe (i.e. a filesystem with strong error detection reports no errors but the MD check action still reports a data/parity mismatch), this is usually more indicative of a lost write during a re-write operation (e.g. the machine was powered off in the middle of updating the contents of a stripe), as the parity is written last -- i.e. the parity would be for the old data in that stripe, not the data as it is now.

The MD "repair" action (if you are ABSOLUTELY CERTAIN that it is the parity that is bad) will automatically correct this problem, which you should do, as the failure of a disk containing a data block within that stripe will then leave you with incorrectly calculated data that will then start showing up as filesystem errors (if you're fortunate enough to be using such a filesystem).

Of course all of the usual caveats about checking SMART statistics apply in determining whether a drive is still suitable for continued use. If the same drive kept showing up with the same problems, I'd retire it; if the drive starts reporting an increase in reallocated sector count, I'd retire it; and so on.

Re: Bcachefs Goes to "Externally Maintained"

#394

Post by tuna74 »

p_l wrote: post #322
messe wrote: post #312

Earlier quoted context omitted.

I think you might have missed the point. Yes. Oracle have that copyright. That's the whole fucking point. Anything from before the fork is still licensed (and pretty much everything after) is still under the CDDL which is possibly in conflict with the GPL.

Oracle can't do anything. They can't relicense code that was already released as CDDL in any form other than what they did when they closed down Solaris. The CDDL being unacceptable is the same issue that GPL3 or Apache is unacceptable - unlike GPLv2, CDDL mandates patent licensing as far as the code is considered.

Oracle can relicense any code they have copyright on, except for derived code like Linux drivers. They choose not to.

Re: Bcachefs Goes to "Externally Maintained"

#395

Post by rekoil »

motorest wrote: post #306
nullc wrote: post #287

Earlier quoted context omitted.

> It's a fireable offense in most companies. In a company there are other employees who have your success as part of their job function. People to train you, to talk you down off a ledge, people to step in and guard you against misunderstanding or criticisms. People to advocate for you or send you home before a dispute crosses a point of no return. You're also paid to be there, to put up with the companies BS, .. the…

> In a company there are other employees who have your success as part of their job function. Yes, and they enforce basic relase processes to ensure you don't break releases by skipping QA processes or introducing untested and unverified features in release candidates. And you sure as hell don't have primadonna developers stay in the payroll for long if they start throwing tantrums and personal attacks towards fellow…

> And you sure as hell don't have primadonna developers stay in the payroll for long if they start throwing tantrums and personal attacks towards fellow engineers when they are asked to follow the release process or are called out for trying to sneak untested changes in mission-critical components.

You've obviously not been in this industry for very long, about half the places I've been to have had at least one "rockstar developer" that was actually quite mediocre but had the ear of management which thought they were all that and as a result got to do whatever they wanted.

Re: Bcachefs Goes to "Externally Maintained"

#396

Post by rekoil »

solarkraft wrote: post #362
koverstreet wrote: post #286

Earlier quoted context omitted.

That's because it is my project, and my responsibility. I can't be bowing to the demands of any one person; I have to balance the wants and needs of everyone and prioritize shipping something that works above all else. Repeatedly we've seen that those priorities are not shared, unfortunately. Arguments are just as heated as they ever were, but now instead of arguing over the actual issues - does this work, are we doi…

> But my core mission is just shipping a reliable trustworthy filesystem You’re very successful at building the file system, but your attempt at shipping it appears to have largely failed. This is a tragic bummer. Even if working with Linus is hard (and even if he is behaving irrationally), it’s what would’ve allowed you to get bcachefs in hundreds of millions of users’ hands. This seems worthy of a compromise consid…

> Now almost nobody will be able to use it.

Which hurts so much because it truly seems amazing from so many perspectives, and I admire Kents dedication to it. I was extremely excited to make a large bcachefs filesystem, with RAID6-like redundancy, foreground NVMe disks and spinning rust as background storage, and it seemed to be in the near future too, but now it probably won't happen at all which makes me incredibly sad for us all.

Re: Bcachefs Goes to "Externally Maintained"

#397

Post by rekoil »

koverstreet wrote: post #365
jjaksic wrote: post #351

Earlier quoted context omitted.

Linus is not your boss in the sense that he pays you and can tell you what you do day o day, but he is your "boss" in the sense that he's the one who ultimately approves your work (which includes both your code and your conduct). It's a "two-way street" (you can walk away as much as he), but you need to understand that this is not an equal relationship. It might have been if Linux did not yet have a file system and y…

No, it's not an equal relationship. Linus and the kernel do not have a great track record on filesystems :) XFS has been burning through maintainers, and btrfs never fully stabilized. Given that, it doesn't make sense for anyone to be trying to dictate; that's a track record that should be cause for reexamining how we do things. Working with the kernel has been extremely disruptive to bcachefs development and the com…

> Working with the kernel has been extremely disruptive to bcachefs development and the community

From an excited and hopeful potential future user looking in, this sounds like "working with has been extremely disruptive to me selling my produce", there's nowhere else your customers can realistically get your product, unless you want to only sell to extremely picky customers who will make the hour long trip out to your farm and back.

> I do not strictly need bcachefs to be in the kernel.

But I (and your other users) do...

I shouldn't be weighing in, I'm a lightweight with no skin in the game here, just a regular (very technical) user. I guess I just want you to know that your product has many people that want to use it, but even I won't drive to your farm for it, as I can't risk a Linux-update breaking the DKMS-modules that make my system able to use the bcachefs filesystem all my data is on.

I don't know what possible avenue we have of getting bcachefs back into the kernel and maintained, but I hope you find it, whatever it is.

Re: Bcachefs Goes to "Externally Maintained"

#398

Post by koverstreet »

rekoil wrote: post #397
koverstreet wrote: post #365

Earlier quoted context omitted.

No, it's not an equal relationship. Linus and the kernel do not have a great track record on filesystems :) XFS has been burning through maintainers, and btrfs never fully stabilized. Given that, it doesn't make sense for anyone to be trying to dictate; that's a track record that should be cause for reexamining how we do things. Working with the kernel has been extremely disruptive to bcachefs development and the com…

> Working with the kernel has been extremely disruptive to bcachefs development and the community From an excited and hopeful potential future user looking in, this sounds like "working with has been extremely disruptive to me selling my produce", there's nowhere else your customers can realistically get your product, unless you want to only sell to extremely picky customers who will make the hour long trip out to yo…

I'm not trying to do this as fast as possible and get it out to every single user as fast as possible

That's tech industry thinking; that is what btrfs did :)

In the long run, slow is fast and wins the race. Being in the kernel would have been great if it grew the development community, but instead the opposite happened - it drove people way.

The important thing is to get it done, with all the reliability and hardening and features that people want. There's no reason it can't go back in later.

Re: Bcachefs Goes to "Externally Maintained"

#399

Post by rekoil »

koverstreet wrote: post #398
rekoil wrote: post #397

Earlier quoted context omitted.

> Working with the kernel has been extremely disruptive to bcachefs development and the community From an excited and hopeful potential future user looking in, this sounds like "working with has been extremely disruptive to me selling my produce", there's nowhere else your customers can realistically get your product, unless you want to only sell to extremely picky customers who will make the hour long trip out to yo…

I'm not trying to do this as fast as possible and get it out to every single user as fast as possible That's tech industry thinking; that is what btrfs did :) In the long run, slow is fast and wins the race. Being in the kernel would have been great if it grew the development community, but instead the opposite happened - it drove people way. The important thing is to get it done, with all the reliability and hardeni…

I mean I've read a lot of what you've posted everywhere, and I can't really say I disagree with much of it, it's just a shame it all happened the way it did.

Hope "go back in later" isn't too far down the line, I have 10x8TB spinners, and 4x2TB NVMe disks that I'm looking to move from md-raid + BTRFS to something else, and I really want that to be an in-tree filesystem.

Re: Bcachefs Goes to "Externally Maintained"

#400

Post by pas »

koverstreet wrote: post #106
arp242 wrote: post #84

Earlier quoted context omitted.

Kent brings up Debian himself, unprompted. This is one of the problems: Kent is frequently unable to accept that things don't go his way. He will keep bringing it up again and again and he just grinds people down with it. If you see just one bit of it then it may seem somewhat reasonable, but it's really not because this is the umpteenth time this exact discussion is happening and it's groundhog day once again. This…

You do realize that data integrity issues are not "live and let live" type things, right? And there's a real connection to the issue that sparked all this drama in the kernel and the Debian drama: critical system components (the kernel, the filesystem, and others) absolutely need to be able to get bugfixes in a timely manner. That's not optional. With Debian, we had a package maintainer who decided that unbundling Ru…

... yes, yes, they fucking are.

It's the people we meet along (and get along with) not the fucking data.

Cherish people who want to work with you, not one more byte saved.

Users are responsible for their own backups. If you want to be really responsible educate them. (As you already helped some of them to build their own kernel, you can nudge them to keep proper backups too!)

If you have problems with downstream being slow then offer workarounds.

How ridiculous is all this, really? Instead of hosting a shell script that sets up a PPA or a cron job with updates or whatever you try to brute force things through Debian and Linus?

If users are that important for you give them your phone number or whatever.

Seriously. Based on all of what you wrote you need to put effort into having a direct line to your users. (Which is completely fine nowadays. Discord and Twitch/YT office hours and whatnot are all the hype nowadays.)

Stop projecting your needs onto other maintainers.

> critical system components (the kernel, the filesystem, and others) absolutely need to be able to get bugfixes in a timely manner. That's not optional.

that's not how these projects are set up. (and even though your code is upstream to them, your users were (are) downstream to them. they were hosting your project for their users, they can (and did) decide to stop hosting your project.)

data integrity and data-loss-prevention are not considered security updates - where we have a culture of out of bad updates. (because the workaround for them is the standing order is to have working backups.)

Post reply on HN