Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Bcachefs Goes to "Externally Maintained"

lwn.net

Post reply on HN
Print view View original

231–240 of 400 posts

Re: Bcachefs Goes to "Externally Maintained"

#231

Post by koverstreet »

motorest wrote: post #173
koverstreet wrote: post #105

Earlier quoted context omitted.

When rules and authority start to take precedence over making sure things work, things have gone off the rails and we're not doing engineering anymore.

> When rules and authority start to take precedence over making sure things work, (...) Didn't Linus lambast you for "lack of testing and collaboration before submitting patches", to the point the patches you were trying to push weren't even building? https://ostechnix.com/linus-torvalds-expresses-frustration-w...

Linus has broken the build more recently than I have. (In the time since bcachefs went upstream, we've both done that once, that I've seen).

Linus doesn't seem to believe in automated testing. He just seems to think that there's no way I could QA code as quickly as I do, but that's because I've invested heavily in automated testing and building up a community of people doing very good testing and QA work; bcachefs's automated testing is the best of any upstream filesystem that I've seen (there's a whole cluster of machines dedicated to this), and I have people running my latest branch on a daily basis.

Nearly all of the collaboration just happens on IRC.

For big changes I wait for explicit acks from testers that they've ran it and things look good; a lot of people read and review my code too, it's just typically less formal than the rest of the kernel.

Re: Bcachefs Goes to "Externally Maintained"

#232

Post by koverstreet »

motorest wrote: post #174
ranger_danger wrote: post #124

Earlier quoted context omitted.

I think this attitude is exactly why this happened. I would have done the same thing. Do you argue with your school teachers that your book report shouldn't be due on Friday because it's not perfect yet? I read several of your response threads across different websites. The most interesting to me was LWN, about the debian tools, where an actual psychologist got involved. All the discussions seem to show the same issu…

> All the discussions seem to show the same issue: You disagree with policies held by people higher up than you, and you struggle with respecting their decisions and moving on. I think it's less subtle than that. The straw that broke the camel's back was quite literally abuse towards other kernel developers. https://lwn.net/Articles/999197/

You might want to read the full story on that one.

Re: Bcachefs Goes to "Externally Maintained"

#233

Post by thoroughburro »

bigstrat2003 wrote: post #182
cmurf wrote: post #29

Earlier quoted context omitted.

Absurd to claim it’s unusable without any qualification whatsoever. Single, dup, raid0, raid1, raid10 have been usable and stable for a decade or more.

I lost my BTRFS RAID-1 array a year or two ago when one of my drives went offline. Just poof, data gone and I had to rebuild. I am not saying that it happens all the time, but I wouldn't say it's completely bulletproof either.

What did you try before giving up?

All the anecdotes I see tend to be “my drive didn’t mount, and I tried nothing before giving up because everyone knows BTRFS sux lol”. My professional experience meanwhile is that I’ve never once been able to not (very easily!) recover a BTRFS drive someone else has given up for dead… just by running its standard recovery tools.

Re: Bcachefs Goes to "Externally Maintained"

#234

Post by motorest »

charcircuit wrote: post #217
motorest wrote: post #214

Earlier quoted context omitted.

> This was a bug fix. I'm not sure exactly what you are talking about, and I'm not sure you do either. The discussion that preceded bcachefs to be dropped from the Linux kernel mainline involved an attempt to sneak a new features in RC, sidestepping testing and QA work, which was followed up by yet more egregious behavior from the mantainer. https://www.phoronix.com/news/Linux-616-Bcachefs-Late-Featur...

>sneak a new features in RC Too solve a bug with the filesystem that people in the wild were hitting. Like how Linus has said in the past with how there is a blurry line between security fixes and bug fixes. There is a blurry line between filesystem bugs and recovery features. If you read the email it is clear that the full feature has more work needed and this is more of a basic implementation to address bugs that p…

> Too solve a bug with the filesystem that people in the wild were hitting.

So you acknowledge that this last episode involved trying to push new features into a RC.

As it was made abundantly clear, not only is the point of RC branches to only get tiny bugfixes after testing, the feature work that was presented was also untested and risked introducing major regressions.

All these red flags were repeatedly raised in the mailing list by multiple kernel maintainers. Somehow you're ignoring all the feedback and warnings and complains raised by people from Linux kernel maintainers, and instead you've opted to try to gaslight the thread.

Re: Bcachefs Goes to "Externally Maintained"

#235

Post by motorest »

koverstreet wrote: post #232
motorest wrote: post #174

Earlier quoted context omitted.

> All the discussions seem to show the same issue: You disagree with policies held by people higher up than you, and you struggle with respecting their decisions and moving on. I think it's less subtle than that. The straw that broke the camel's back was quite literally abuse towards other kernel developers. https://lwn.net/Articles/999197/

You might want to read the full story on that one.

> You might want to read the full story on that one.

I read the full story. Everyone else can do the same. Somehow it seems you opt to skip it and prefer to be deeply invested in creating an alternative reality.

Re: Bcachefs Goes to "Externally Maintained"

#236

Post by koverstreet »

Icathian wrote: post #221
koverstreet wrote: post #159

Earlier quoted context omitted.

that would be bcachefs :) It's an entirely clean slate design, and I spent years taking my time on the core planning out the design; it's as close to perfect as I can make it. The only things I can think of that I would change or add given unlimited time and budget: - It should be written in Rust, and even better a Rust + dependent types (which I suspect could be done with proc macros) for formal verification. And ca…

I happen to work at a company that uses a ton of capnp internally and this is the first time I've seen it mentioned much outside of here. Would you mind describing what about it you think would make it a good fit for something like bcachefs?

Cap'n proto is basically a schema language that gets you a well defined in-memory representation that's just as good as if you were writing C structs by hand (laboriously avoiding silent padding, carefully using types with well defined sizes) - without all the silent pitfalls of doing it manually in C.

It's extremely well thought out, it's minimalist in all the right ways; I've found the features and optimizations it has to be things that are borne out of real experience that you would want end up building yourself in any real world system.

E.g. it gives you the ability to add new fields without breaking compatibility. That's the right way to approach forwards/backwards compatibility, and it's what I do in bcachefs and if we'd been able to just use cap'n proto it would've taken out a lot of manual fiddly work.

The only blocker to using it more widely in my own code is that it's not sufficiently ergonomic in Rust - Rust needs lenses, from Swift.

Re: Bcachefs Goes to "Externally Maintained"

#237

Post by nullc »

koverstreet wrote: post #159
TheAceOfHearts wrote: post #147

Earlier quoted context omitted.

> But there's a ton of room for improvement beyond what ZFS did. Say more? I can't say I've really thought that much about filesystems and I'm curious in what direction you think they could be taken if time and budget weren't an issue.

that would be bcachefs :) It's an entirely clean slate design, and I spent years taking my time on the core planning out the design; it's as close to perfect as I can make it. The only things I can think of that I would change or add given unlimited time and budget: - It should be written in Rust, and even better a Rust + dependent types (which I suspect could be done with proc macros) for formal verification. And ca…

> - Erasure coding is much more performant than ZFS's

any plans for much lower rates than typical raid?

Increasingly modern high density devices are having block level failures at non-trivial rates instead of or in addition to whole device failures. A file might be 100,000 blocks long, adding 1000 blocks of FEC would expand it 1% but add tremendous protection against block errors. And can do so even if you have a single piece of media. Doesn't protect against device failures, sure, though without good block level protection device level protection is dicey since hitting some block level error when down to minimal devices seems inevitable and having to add more and more redundant devices is quite costly.

Re: Bcachefs Goes to "Externally Maintained"

#238

Post by righthand »

koverstreet wrote: post #231
motorest wrote: post #173

Earlier quoted context omitted.

> When rules and authority start to take precedence over making sure things work, (...) Didn't Linus lambast you for "lack of testing and collaboration before submitting patches", to the point the patches you were trying to push weren't even building? https://ostechnix.com/linus-torvalds-expresses-frustration-w...

Linus has broken the build more recently than I have. (In the time since bcachefs went upstream, we've both done that once, that I've seen). Linus doesn't seem to believe in automated testing. He just seems to think that there's no way I could QA code as quickly as I do, but that's because I've invested heavily in automated testing and building up a community of people doing very good testing and QA work; bcachefs's…

Yeah but you don’t get to make the calls. Linus does and your “well kernel daddy does it too” and “actually I’m doing it better than my critics understand” don’t play well with the kernel daddy (or really any bdfl). Do you not see your comment as dismissive?

All your comments are dismissive of the criticisms so far and you’re shrugging your shoulders as to why.

It’s great you’re able to reason and defend yourself but Linux as a whole is larger than you and refusing to submit to their ways will make technology move no where.

Re: Bcachefs Goes to "Externally Maintained"

#239

Post by jeltz »

JackSlateur wrote: post #222
koverstreet wrote: post #126

Earlier quoted context omitted.

It's a bugfix, and bugfixes are allowed at and time - weighing regression risk against where we're at in the cycle. It was a very high severity bug, low regression risk for the fix, and we were at rc3.

Reading https://lore.kernel.org/all/4xkggoquxqprvphz2hwnir7nnuygeybf... It is a not a bugfix, and you know it :( If you are not acting on bad faith, I suggest you read Wittgensen He has made a lot of work around the idea of language, which basically boil down to the fact that words have no intrinsic meaning : the meaning of a word is the meaning that a given population gives to that word So in your case, you may be r…

> - New option: journal_rewind [...]

> - Some new btree iterator tracepoints, for tracking down some livelock-ish behaviour we've been seeing in the main data write path.

Yeah, how are these two things bug-fixes? Especially the first one should not be merged late.

Re: Bcachefs Goes to "Externally Maintained"

#240

Post by nullc »

mort96 wrote: post #224
koverstreet wrote: post #101

Earlier quoted context omitted.

No, the problem wasn't following the rules. The patch that kicked off the current conflict was the 'journal_rewind' patch; we recently (6.15) had the worst bug in the entire history upstream - it was taking out entire subvolumes. The third report got me a metadata dump with everything I needed to debug the issue, thank god, and now we have a great deal of hardening to ensure a bug like this can never happen again. Su…

It's so sad to see an excellent engineer such as yourself, building what seems like an excellent filesystem that has the potential to be better than everything else available for Linux for many use cases, completely fail to achieve your goals because you lack the people skills to navigate working as a part of a team under a technical leader. Every comment and e-mail I've seen from you has demonstrated an impressive l…

> minimum avoid pissing everyone else off

Which also, at times, means appeasing people even when you are confident that they are wrong because you need their cooperation in the future. In a large complicated system, being able to work together is often more important to the system's reliability, performance, etc. than being as right as possible.

Plus even when you're confident you are in the right you might still be in the wrong. After all, the people you are disagreeing with are also superbly competent and they believe they're in the right just as you do. There can be hills worth dying on, but they ought to be very rare.

Post reply on HN