Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Bcachefs Goes to "Externally Maintained"

lwn.net

Post reply on HN
Print view View original

261–270 of 400 posts

Re: Bcachefs Goes to "Externally Maintained"

#261

Post by mort96 »

nullc wrote: post #240
mort96 wrote: post #224

Earlier quoted context omitted.

It's so sad to see an excellent engineer such as yourself, building what seems like an excellent filesystem that has the potential to be better than everything else available for Linux for many use cases, completely fail to achieve your goals because you lack the people skills to navigate working as a part of a team under a technical leader. Every comment and e-mail I've seen from you has demonstrated an impressive l…

> minimum avoid pissing everyone else off Which also, at times, means appeasing people even when you are confident that they are wrong because you need their cooperation in the future. In a large complicated system, being able to work together is often more important to the system's reliability, performance, etc. than being as right as possible. Plus even when you're confident you are in the right you might still be…

Exactly. An extremely important part of working in some hierarchical organizational structure, be that as a Linux kernel developer or as an employee at a company, is the ability to disagree with a superior's decision yet acquiesce and go along with it. Good organizations leave room for disagreement, but there always comes a point where someone in a leadership position has made a final decision and the time for debate is over.

Re: Bcachefs Goes to "Externally Maintained"

#262

Post by motorest »

koverstreet wrote: post #231
motorest wrote: post #173

Earlier quoted context omitted.

> When rules and authority start to take precedence over making sure things work, (...) Didn't Linus lambast you for "lack of testing and collaboration before submitting patches", to the point the patches you were trying to push weren't even building? https://ostechnix.com/linus-torvalds-expresses-frustration-w...

Linus has broken the build more recently than I have. (In the time since bcachefs went upstream, we've both done that once, that I've seen). Linus doesn't seem to believe in automated testing. He just seems to think that there's no way I could QA code as quickly as I do, but that's because I've invested heavily in automated testing and building up a community of people doing very good testing and QA work; bcachefs's…

> Linus has broken the build more recently than I have.

Even taking your claims at face value (which from this thread alone is a heck of a leap) I'm baffled by the way you believe this holds any relevance.

I mean, the kernel project has in place a quality assurance process designed to minimize the odds of introducing problems when preparing a release. You were caught purposely ignoring any QA process in place and trying to circumvent the whole quality assurance process and sneak into a RC features that were untested and unverified.

There is a QA process, and you purposely decided to ignore it and plow away. And then your best argument for purposely ignoring any semblance of QA is that others may or may not have broken a build before?

Come on, man. You know better than this. How desperate are you to avoid any accountability to pull these gaslighting stunts?

Re: Bcachefs Goes to "Externally Maintained"

#263

Post by [deleted] »

yjftsjthsd-h wrote: post #93
ofrzeta wrote: post #28

Earlier quoted context omitted.

Suse Linux Enterprise still uses Btrfs as the Root-FS, so it can't be that bad, right? What is Chris Mason actually doing these days? I did some googling and only found out that he was working on a tool called "rsched".

I used btrfs a few years ago, on OpenSUSE, because I also thought that would work, and it was on a single disk. It lost my root filesystem twice.

[deleted]

Re: Bcachefs Goes to "Externally Maintained"

#264

Post by koverstreet »

nullc wrote: post #237
koverstreet wrote: post #159

Earlier quoted context omitted.

that would be bcachefs :) It's an entirely clean slate design, and I spent years taking my time on the core planning out the design; it's as close to perfect as I can make it. The only things I can think of that I would change or add given unlimited time and budget: - It should be written in Rust, and even better a Rust + dependent types (which I suspect could be done with proc macros) for formal verification. And ca…

> - Erasure coding is much more performant than ZFS's any plans for much lower rates than typical raid? Increasingly modern high density devices are having block level failures at non-trivial rates instead of or in addition to whole device failures. A file might be 100,000 blocks long, adding 1000 blocks of FEC would expand it 1% but add tremendous protection against block errors. And can do so even if you have a sin…

It's been talked about. I've seen some interesting work to use just a normal checksum to correct single bit errors.

If there's an optimized implementation we can use in the kernel, I'd love to add it. Even on modern hardware, we do see bit corruption in the wild, it would add real value.

Re: Bcachefs Goes to "Externally Maintained"

#265

Post by nullc »

koverstreet wrote: post #264
nullc wrote: post #237

Earlier quoted context omitted.

> - Erasure coding is much more performant than ZFS's any plans for much lower rates than typical raid? Increasingly modern high density devices are having block level failures at non-trivial rates instead of or in addition to whole device failures. A file might be 100,000 blocks long, adding 1000 blocks of FEC would expand it 1% but add tremendous protection against block errors. And can do so even if you have a sin…

It's been talked about. I've seen some interesting work to use just a normal checksum to correct single bit errors. If there's an optimized implementation we can use in the kernel, I'd love to add it. Even on modern hardware, we do see bit corruption in the wild, it would add real value.

It's pretty straight forward to use a normal checksum to correct single or even more bit errors (depending on the block size, choice of checksum, etc). Though I expect those bit errors are bus/ram, and hopefully usually transient. If there is corruption on the media, the whole block is usually going to be lost because any corruptions means that its internal block level FEC has more errors than it can fix.

I was more thinking along the lines of adding dozens or hundreds of correction blocks to a whole file, along the lines of par (though there are much faster techniques now).

Re: Bcachefs Goes to "Externally Maintained"

#266

Post by koverstreet »

p_l wrote: post #229
koverstreet wrote: post #117

Earlier quoted context omitted.

I can't speak with any authority on ZFS, I know its structure the least out of all the major filesystems. I do a ton of reading through forums gathering user input, and lots of people chime in with stories of lost filesystems. I've seen reports of lost filesystems with ZFS and I want to say I've seen them at around the same frequency of XFS; both are very rare. My concern with ZFS is that they seem to have taken the…

The lack of traditional 'fsck' is because its operation would be exact same as normal driver operation. The most extreme case involves a very obscure option that lets you explicitly rewind transactions to one you specify, which I've seen used to recover a broken driver upgrade that led to filesystem corruption in ways that most FSCK just barf on, including XFS' For low-level meddling and recovery, there's a filesyste…

Rewinding transactions is cool. Bcachefs has that too :)

What happens on ZFS if you lose all your alloc info? Or are there other single points of failure besides the ublock in the on disk format?

Re: Bcachefs Goes to "Externally Maintained"

#267

Post by aaronmdjones »

zitsarethecure wrote: post #219
arccy wrote: post #203

Earlier quoted context omitted.

people understand they're different, but if bcachefs is out, then that leaves btrfs as the only modern in-tree filesystem, but apparently you can't trust it with important data either.

I've been using btrfs on my NAS for years and have not had any problems. I suspect there are a hell of a lot of people like me you will not hear about because people don't generally get as vocal when things just work.

The venn diagram of "people who want a modern copy-on-write filesystem with snapshots to manage large quantities of data" and "people who want a massive pool of fault-tolerant storage" (e.g. building a NAS) has some pretty significant overlap.

The latter is where BTRFS is still hobbled: While the RAID-0, RAID-1, & RAID-10 modes work absolutely fine, the RAID-5 & RAID-6 modes are still broken, with an explicit warning during mkfs time (and in the manpages) that the feature is still experimental and should not be used to hold data that you care about retaining. This has, and continues to, bite people, with terabytes of data loss (backups are important, people!). That then sours them on every other aspect of ever using BTRFS again.

Re: Bcachefs Goes to "Externally Maintained"

#268

Post by quotemstr »

nirava wrote: post #251
quotemstr wrote: post #168

Earlier quoted context omitted.

> Being correct comes second to being agreeable in human-human interactions Prioritizing agreeableness above correctness is the reason the space shuttle Challenger blew up. The bcachefs fracas is interesting and important because it's like a stain making some damn germ's organelles visible: it highlights a psychological division in tech and humanity in general between people who prioritize 1) deferring to authority,…

Ugh, this is a lot of words for nothing. 1. I laid down what I perceived as the state of things. The generalizations I drew from observing the system that is Linux development. Nowhere have I prescribed that kent "follow" my ideas. Simply that he can use these to try to understand the unfairness he feels. 2. Your anarcho-individualistic development ideas sound good in theory, but if they ever worked in practice we mi…

> Simply that he can use these to try to understand the unfairness he feels.

You're suggesting he deal with unfairness by internalizing it as virtue? That's how to make people who cheer at other people's failures.

> Your anarcho-individualistic development ideas sound good in theory

Thanks for illustrating my point. No project, >3 or <= 3, has ever made any new technology by adopting as a tenet that social agreement inside the project is more important than correctly modeling the world outside it, and you're suggesting I'm using inefficiently agreeable-sounding words to express it.

Re: Bcachefs Goes to "Externally Maintained"

#269

Post by koverstreet »

nullc wrote: post #265
koverstreet wrote: post #264

Earlier quoted context omitted.

It's been talked about. I've seen some interesting work to use just a normal checksum to correct single bit errors. If there's an optimized implementation we can use in the kernel, I'd love to add it. Even on modern hardware, we do see bit corruption in the wild, it would add real value.

It's pretty straight forward to use a normal checksum to correct single or even more bit errors (depending on the block size, choice of checksum, etc). Though I expect those bit errors are bus/ram, and hopefully usually transient. If there is corruption on the media, the whole block is usually going to be lost because any corruptions means that its internal block level FEC has more errors than it can fix. I was more…

You'd think that, wouldn't you? But there are enough moving parts in the IO stack below the filesystem that we do see bit errors. I don't have enough data to do correlations and tell you likely causes, but they do happen.

I think SSDs are generally worse than spinning rust (especially enterprise grade SCSI kit), the hard drive vendors have been at this a lot longer and SSDs are massively more complicated. From the conversations I've had with SSD vendors, I don't think they've put the some level of effort into making things as bulletproof as possible yet.

Re: Bcachefs Goes to "Externally Maintained"

#270

Post by ZenoArrow »

koverstreet wrote: post #159
TheAceOfHearts wrote: post #147

Earlier quoted context omitted.

> But there's a ton of room for improvement beyond what ZFS did. Say more? I can't say I've really thought that much about filesystems and I'm curious in what direction you think they could be taken if time and budget weren't an issue.

that would be bcachefs :) It's an entirely clean slate design, and I spent years taking my time on the core planning out the design; it's as close to perfect as I can make it. The only things I can think of that I would change or add given unlimited time and budget: - It should be written in Rust, and even better a Rust + dependent types (which I suspect could be done with proc macros) for formal verification. And ca…

> Closest realization of "filesystem as a database" that I know of

More so than BFS?

https://en.m.wikipedia.org/wiki/Be_File_System

"Like its predecessor, OFS (Old Be File System, written by Benoit Schillings - formerly BFS), it includes support for extended file attributes (metadata), with indexing and querying characteristics to provide functionality similar to that of a relational database."

Post reply on HN