Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Bcachefs Goes to "Externally Maintained"

lwn.net

Post reply on HN
Print view View original

271–280 of 400 posts

Re: Bcachefs Goes to "Externally Maintained"

#271

Post by sc68cal »

mort96 wrote: post #255
koverstreet wrote: post #246

Earlier quoted context omitted.

I get a ton of comments like this. Pointing the finger at the skills I lack and my inability, while ignoring the wider picture, of the kernel burning out maintainers and not doing well on filesystems. It's wearying.

You get a ton of comments like this because it's true. There are real problems in the kernel, I've seen how hostile it can be to people who are just trying to do the right thing and upstream their changes etc. But your case isn't that. Your behavior would get you in trouble at any job where you have to follow rules set by other people. Your refusal to treat your part of the kernel as anything other than your personal…

Kent has gotten this same feedback across practically every single platform that has discussed his issues. He is unable to take critique and will instead just continue to argue and be combative, therefore proving yet again why he is in this situation in the first place

Re: Bcachefs Goes to "Externally Maintained"

#272

Post by motorest »

nullc wrote: post #240
mort96 wrote: post #224

Earlier quoted context omitted.

It's so sad to see an excellent engineer such as yourself, building what seems like an excellent filesystem that has the potential to be better than everything else available for Linux for many use cases, completely fail to achieve your goals because you lack the people skills to navigate working as a part of a team under a technical leader. Every comment and e-mail I've seen from you has demonstrated an impressive l…

> minimum avoid pissing everyone else off Which also, at times, means appeasing people even when you are confident that they are wrong because you need their cooperation in the future. In a large complicated system, being able to work together is often more important to the system's reliability, performance, etc. than being as right as possible. Plus even when you're confident you are in the right you might still be…

> Which also, at times, means appeasing people even when you are confident that they are wrong because you need their cooperation in the future.

Being unwilling to follow basic QA processes in preparation of a release candidate, and then doubling down by attacking the release engineer with claims the QA process doesn't apply to you because you know better, is something that is far more serious than lacking basic soft skills. It's a fireable offense in most companies.

Re: Bcachefs Goes to "Externally Maintained"

#273

Post by throwup238 »

JackSlateur wrote: post #222
koverstreet wrote: post #126

Earlier quoted context omitted.

It's a bugfix, and bugfixes are allowed at and time - weighing regression risk against where we're at in the cycle. It was a very high severity bug, low regression risk for the fix, and we were at rc3.

Reading https://lore.kernel.org/all/4xkggoquxqprvphz2hwnir7nnuygeybf... It is a not a bugfix, and you know it :( If you are not acting on bad faith, I suggest you read Wittgensen He has made a lot of work around the idea of language, which basically boil down to the fact that words have no intrinsic meaning : the meaning of a word is the meaning that a given population gives to that word So in your case, you may be r…

I think you mean Wittgenstein, though I wouldn’t recommend Philosophical Investigations as an entrypoint.

Re: Bcachefs Goes to "Externally Maintained"

#274

Post by quotemstr »

nullc wrote: post #256
koverstreet wrote: post #250

Earlier quoted context omitted.

Thanks, I've been struggling to put this into words. When you're working on the core technology we all depend on, correctness is not optional.

Linux is not correct. Linux has never been correct. Linux will never be correct. An incorrect belief that it is correct can only make it less correct. You must know this when it comes to your own work. Why isn't bcachefs written in augmented rust with dependent types and formal correctness proofs for every line of code? How could there ever be a data losing bug if you had a formal proof that the file system could nev…

You're staking out quite the postmodernist position there. All models are wrong, so who's to say that Alice's data corruption is worse than Bob's man page typo? The important thing is we stick to process with a proven track record, right?

I don't buy it. Object level considerations do matter. Alice's bug really is worse than Bob's. That "proven track record" shouldn't apply to Alice, and insisting that it does for the sake of process, in a way indifferent to the facts of the situation, is just a pretext for doing primate social hierarchy deference rituals in a situation in which they're producing a worse outcome and everyone knows it.

Re: Bcachefs Goes to "Externally Maintained"

#275

Post by nullc »

koverstreet wrote: post #269
nullc wrote: post #265

Earlier quoted context omitted.

It's pretty straight forward to use a normal checksum to correct single or even more bit errors (depending on the block size, choice of checksum, etc). Though I expect those bit errors are bus/ram, and hopefully usually transient. If there is corruption on the media, the whole block is usually going to be lost because any corruptions means that its internal block level FEC has more errors than it can fix. I was more…

You'd think that, wouldn't you? But there are enough moving parts in the IO stack below the filesystem that we do see bit errors. I don't have enough data to do correlations and tell you likely causes, but they do happen. I think SSDs are generally worse than spinning rust (especially enterprise grade SCSI kit), the hard drive vendors have been at this a lot longer and SSDs are massively more complicated. From the co…

One thing to keep in mind is that correction always comes as some expense of detection.

Generally a code that can always detect N errors can only always correct N/2 errors. So you detect an errored block, you correct up to N/2 errors. The block now passes but if the block actually had N errors, your correction will be incorrect and you now have silent corruption.

The solution to this is just to have an excess of error correction power and then don't use all of it. But that can be hard to do if you're trying to shoehorn it into an existing 32-bit crc.

How big are the blocks that the CRC units cover in bcachefs?

Re: Bcachefs Goes to "Externally Maintained"

#276

Post by rwmj »

rwmj wrote: post #208
nextaccountic wrote: post #206

Earlier quoted context omitted.

For the Debian thing, I'm not sure on the specifics for bcachefs-progs (I'm going by what the author is reporting and some blog posts) but I think the problem with Debian is that they willfully ignore when upstream says "this is only compatible with this library version 2.1.x" and will downgrade or upgrade the library into not supported versions, to match the versions used in other programs already packaged. This kin…

But that's exactly the point here. In the context of a whole distribution, you don't want to update some package to a new version (on a stable branch), because that would affect lots of other packages that depended on that one. It may even be that other packages cannot work with the new updated dependency. Even if they can, end users don't want versions to change greatly (again, along a stable branch). Upstreams shou…

BTW here's where I ported nbdfuse from v2 to v3 so you can see the kinds of changes: https://gitlab.com/nbdkit/libnbd/-/commit/c74c7d7f01975e708b...

Re: Bcachefs Goes to "Externally Maintained"

#277

Post by yjftsjthsd-h »

goneri wrote: post #145
qalmakka wrote: post #141

The whole situation is moronic at best. Linux needs a decent modern filesystem in tree. ZFS would easily be it, but unfortunately Sun decided back in the '00s to fuck Linux because they wanted to push Solaris instead. Little they knew ZFS ended up being FreeBSD top feature for years. Btrfs is constantly eating people data, it's a bad joke nowadays. Right now on Linux you're basically forced to constantly deal with ou…

Btrfs is NOT constantly eating people data. You have nothing to back this statement. It's widely used and the default filesystem of several distributions. Most of the problems are like for the other filesystem: caused by the hardware. I've been using it for more than 10 years without any problem and enjoy the experience. And like for any filesystem, I backup my data frequently (with btrbk, thanks for asking).

> Btrfs is NOT constantly eating people data. You have nothing to back this statement.

Constantly may be a strong word, but there is a long line of people sharing tales of woe. It's good that it works for you, but that's not a universal experience.

> It's widely used and the default filesystem of several distributions.

As a former user, that's horrifying.

> Most of the problems are like for the other filesystem: caused by the hardware.

The whole point of btrfs over (say) ext4 is that it's supposed to hold up when things don't work.

Re: Bcachefs Goes to "Externally Maintained"

#278

Post by koverstreet »

ZenoArrow wrote: post #270
koverstreet wrote: post #159

Earlier quoted context omitted.

that would be bcachefs :) It's an entirely clean slate design, and I spent years taking my time on the core planning out the design; it's as close to perfect as I can make it. The only things I can think of that I would change or add given unlimited time and budget: - It should be written in Rust, and even better a Rust + dependent types (which I suspect could be done with proc macros) for formal verification. And ca…

> Closest realization of "filesystem as a database" that I know of More so than BFS? https://en.m.wikipedia.org/wiki/Be_File_System "Like its predecessor, OFS (Old Be File System, written by Benoit Schillings - formerly BFS), it includes support for extended file attributes (metadata), with indexing and querying characteristics to provide functionality similar to that of a relational database."

What BFS did is very cool, and I hope to add that to bcachefs someday.

But I'm talking more about the internals than external database functionality; the inner workings are much more fundamental.

bcachefs internally is structured more like a relational database than a traditional Unix filesystem, where everything hangs off the inode. In bcachefs, there's an extents btree (read: table), an inodes btree, a dirents btree, and a whole bunch of others - we're up to 20 (!).

There's transactions, where you can do arbitrary lookups, updates, and then commit, with all the database locking hidden from you; lookups within a transaction see uncommitted updates from that transaction. There's triggers, which are used heavily.

We don't have the full relational model - no SELECT or JOIN, no indices on arbitrary fields like with SQL (but you can do effectively the same thing with triggers, I do it all the time).

All the database/transactional primitives make the rest of the codebase much smaller and cleaner, and make feature development a lot easier than what you'd expect in other filesystems.

Re: Bcachefs Goes to "Externally Maintained"

#279

Post by rob_c »

koverstreet wrote: post #243
dijksterhuis wrote: post #218

Earlier quoted context omitted.

> In practice bcachefs is used in production with real users. If the experimental label prevents critical bug fixes from making it into the kernel then it would be better to just remove that label. alternative perspective: those users have knowingly and willingly put experimental software into production. it was their choice, they were informed of the risk and so the consequences and responsibility are their’s. it’s…

Again: the elephant in the room is that a lot of bcachefs users are using it explicitly because they have lost a lot of data on btrfs, and they've found it to be more trustworthy. This puts us all in a shitty situation. I want the experimental label to come off at the right time - when every critical bug is fixed and it's as trustworthy as I can reasonably make it, when I know according to the data I have that everyo…

>users are using it explicitly because they have lost a lot of data on btrfs

PLEASE, honestly, EDUCATE THESE USERS. This is still marked experimental for numerous reasons regardless of the 'planned work for 6.18'. Users who can't suffer any data loss and are repeating their mistake of using btrfs shouldn't be using a none default/standard/hardened filesystem period.

Re: Bcachefs Goes to "Externally Maintained"

#280

Post by koverstreet »

motorest wrote: post #262
koverstreet wrote: post #231

Earlier quoted context omitted.

Linus has broken the build more recently than I have. (In the time since bcachefs went upstream, we've both done that once, that I've seen). Linus doesn't seem to believe in automated testing. He just seems to think that there's no way I could QA code as quickly as I do, but that's because I've invested heavily in automated testing and building up a community of people doing very good testing and QA work; bcachefs's…

> Linus has broken the build more recently than I have. Even taking your claims at face value (which from this thread alone is a heck of a leap) I'm baffled by the way you believe this holds any relevance. I mean, the kernel project has in place a quality assurance process designed to minimize the odds of introducing problems when preparing a release. You were caught purposely ignoring any QA process in place and try…

Please, tell us about these wonderful QA processes the kernel has.
Post reply on HN