Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Bcachefs Goes to "Externally Maintained"

lwn.net

Post reply on HN
Print view View original

381–390 of 400 posts

Re: Bcachefs Goes to "Externally Maintained"

#381

Post by bcrl »

immibis wrote: post #342
koverstreet wrote: post #328

Earlier quoted context omitted.

We're not talking about situations where the bcachefs changes could plausibly affect the rest of the kernel, and I am well known even within the kernel community for being on top of potential issues and responsive on bugs.

The merge window policy is not limited to cases where the new bcachefs features affect the rest of the kernel.

Read the following commit log:

    Pull btrfs fixes from David Sterba:
     "Several zoned mode fixes, mount option printing fixups, folio state
      handling fixes and one log replay fix.
    
       - zoned mode:
           - zone activation and finish fixes
           - block group reservation fixes
    
       - mount option fixes:
           - bring back printing of mount options with key=value that got
             accidentally dropped during mount option parsing in 6.8
           - fix inverse logic or typos when printing nodatasum/nodatacow
    
       - folio status fixes:
           - writeback fixes in zoned mode
           - properly reset dirty/writeback if submission fails
           - properly handle TOWRITE xarray mark/tag
    
       - do not set mtime/ctime to current time when unlinking for log
         replay"
Sounds like they added a feature back in that was removed 10 major releases ago. Why is that not considered a feature? By this logic that btrfs merge request should have been rejected.

The "feature" label is messy. If the "feature" means that a user doesn't lose their data in a filesystem, is it really a "feature" or is it a bug fix? It's very much shades of grey.

What a lot of people are missing here is that bcachefs has a lot of test cases for regressions that are run as part of Kent's continuous integration infrastructure. A lot of Linux filesystems do not have this kind of easy regression testing, in part because this kind of testing is easiest when you do it as part of development, but very difficult to achieve any scale of coverage if it's done after the fact. The older the code is in Linux, the more likely it is that it tends to get coverage via the stress tests and intregration tests that various folks run on kernel releases. But such tests are not regression tests for data structures, I/O failures, corrupted data...

So much of what is going on is bikeshedding the process of submitting changes. It would be far more productive for people to look at the process Kent follows during development, and to then make educated comments about the code that arises from that process.

Good developers know the value of tests. Great developers know that improving your test coverage with every bit of new development means you can develop faster over the long run by not getting bogged down by silent regressions.

Edit: I do have to shout out to the XFS folks. xfstests is a huge contributor to the improvement of Linux filesystems over the last few decades.

Re: Bcachefs Goes to "Externally Maintained"

#382

Post by procaryote »

aaronmdjones wrote: post #267
zitsarethecure wrote: post #219

Earlier quoted context omitted.

I've been using btrfs on my NAS for years and have not had any problems. I suspect there are a hell of a lot of people like me you will not hear about because people don't generally get as vocal when things just work.

The venn diagram of "people who want a modern copy-on-write filesystem with snapshots to manage large quantities of data" and "people who want a massive pool of fault-tolerant storage" (e.g. building a NAS) has some pretty significant overlap. The latter is where BTRFS is still hobbled: While the RAID-0, RAID-1, & RAID-10 modes work absolutely fine, the RAID-5 & RAID-6 modes are still broken, with an explicit warning…

If you ignore explicit warnings at mkfs time and then get upset the warning was accurate, you can't really fully blame the file system for it.

Just raid on a lower layer and btrfs on top.

Re: Bcachefs Goes to "Externally Maintained"

#383

Post by Trixter »

wtallis wrote: post #348
Filligree wrote: post #242

Earlier quoted context omitted.

That’s nice and all, but I have five disks in my server. I want the 6 mode. In practice RAIDZ2 works great.

In the case of five disks of the same capacity, RAID6 or RAIDZ2 only gets you 20% more capacity than btrfs RAID1. That's not exactly a huge disparity, usually not enough to be a show-stopper on its own. There are plenty of scenarios where the features ZFS has which btrfs lacks are more important than the features that btrfs has which ZFS lacks. My point is simply that btrfs RAID1 has its uses and shouldn't be dismiss…

This is missing one of the finer points of redundancy: With 4 disks, losing two of them could take out a RAID-10 (both halves of a stripe component), but losing two cannot take out a RAID-6.

The fact that btrfs isn't stable after 15+ years for parity setups is, IMO, unreasonable.

Re: Bcachefs Goes to "Externally Maintained"

#384

Post by ekaryotic »

koverstreet wrote: post #336
magicalhippo wrote: post #330

Earlier quoted context omitted.

From what I have read and recall, the same rules do apply. Rather, the disagreement seems to be over what constitutes a feature and what constitutes a bugfix. As I recall, your view is that the repair code is part of the bugfix. However Linus deems it a feature, and thus applied the "no new features outside the merge window" rule. I think Linus is correct here and you are wrong. New code made to repair flaws that pre…

New features go in during RCs all the time. The hard rule you're thinking of doesn't exist, it's all risk vs. reward.

If you had requested your code be rolled back to an earlier stable version as an alternative way of fixing the bug rather than merging unstable code you may have come to a compromise.

Re: Bcachefs Goes to "Externally Maintained"

#385

Post by aaronmdjones »

procaryote wrote: post #382
aaronmdjones wrote: post #267

Earlier quoted context omitted.

The venn diagram of "people who want a modern copy-on-write filesystem with snapshots to manage large quantities of data" and "people who want a massive pool of fault-tolerant storage" (e.g. building a NAS) has some pretty significant overlap. The latter is where BTRFS is still hobbled: While the RAID-0, RAID-1, & RAID-10 modes work absolutely fine, the RAID-5 & RAID-6 modes are still broken, with an explicit warning…

If you ignore explicit warnings at mkfs time and then get upset the warning was accurate, you can't really fully blame the file system for it. Just raid on a lower layer and btrfs on top.

> If you ignore explicit warnings at mkfs time and then get upset the warning was accurate, you can't really fully blame the file system for it.

Oh, no doubt. I agree.

> Just raid on a lower layer and btrfs on top.

That has its own set of problems. The conventional RAID solution on Linux (MD) also has some pretty terrifying corruption edge cases with RAID-5 and RAID-6 (as I explained in [1]) which will bite you if you're not aware of them and how to work around them.

A robust filesystem purpose-built for the task can only really be found in ZFS.

[1] https://news.ycombinator.com/item?id=42915999

Re: Bcachefs Goes to "Externally Maintained"

#386

Post by nolist_policy »

bcrl wrote: post #381
immibis wrote: post #342

Earlier quoted context omitted.

The merge window policy is not limited to cases where the new bcachefs features affect the rest of the kernel.

Read the following commit log: Pull btrfs fixes from David Sterba: "Several zoned mode fixes, mount option printing fixups, folio state handling fixes and one log replay fix. - zoned mode: - zone activation and finish fixes - block group reservation fixes - mount option fixes: - bring back printing of mount options with key=value that got accidentally dropped during mount option parsing in 6.8 - fix inverse logic or…

> Sounds like they added a feature back in that was removed 10 major releases ago. Why is that not considered a feature?

It's a single +5 -3 commit:

https://lore.kernel.org/linux-btrfs/20250812180009.1412-2-sa...

Re: Bcachefs Goes to "Externally Maintained"

#387

Post by nh2 »

ajross wrote: post #379
nh2 wrote: post #377

Earlier quoted context omitted.

I believe it is helpful because you can stop an app (such as a DB), FS-snapshot, and then e.g. rsync the snapshot or use any other file based backup tool, and this snapshot is fast and will be correct. Doing the same with a block device snapshot is not so easy.

Again, if your system is "incorrect" having been stopped and snapshotted like that, it is also unsafe vs. power loss , something ZFS cannot save you from. Power loss events are vastly more common than poorly checkpointed database[1] events. [1] FWIW: every database worth being called a "database" has some level of robust journaling with checkpoints internally. I honestly don't know what software you're talking about…

You are conflating Consistency and Durability in a way that is not necessary.

FS snapshotting can be useful to work on files on which fsync() is never called, but for which you need a consistent cross-file view nontheless while the system is online.

Another example is the case of sqlite with default settings as discussed recently (https://news.ycombinator.com/item?id=45005866), where the presence of a file determines whether or not transactions will be missing on next access. Because SQLite does not fsync the file's parent directory by default, transactions would be correctly recorded in by FS snapshotting, but lost in a block device snapshot. Your argument is correct that they would also be lost in power loss, but that does not matter for the fact that there exists a valid way to use that software in some situations where you care about consistency and not about power losses.

This is why having such a feature in file systems is useful.

Re: Bcachefs Goes to "Externally Maintained"

#388

Post by [deleted] »

ajross wrote: post #379
nh2 wrote: post #377

Earlier quoted context omitted.

I believe it is helpful because you can stop an app (such as a DB), FS-snapshot, and then e.g. rsync the snapshot or use any other file based backup tool, and this snapshot is fast and will be correct. Doing the same with a block device snapshot is not so easy.

Again, if your system is "incorrect" having been stopped and snapshotted like that, it is also unsafe vs. power loss , something ZFS cannot save you from. Power loss events are vastly more common than poorly checkpointed database[1] events. [1] FWIW: every database worth being called a "database" has some level of robust journaling with checkpoints internally. I honestly don't know what software you're talking about…

[deleted]

Re: Bcachefs Goes to "Externally Maintained"

#389

Post by procaryote »

aaronmdjones wrote: post #385
procaryote wrote: post #382

Earlier quoted context omitted.

If you ignore explicit warnings at mkfs time and then get upset the warning was accurate, you can't really fully blame the file system for it. Just raid on a lower layer and btrfs on top.

> If you ignore explicit warnings at mkfs time and then get upset the warning was accurate, you can't really fully blame the file system for it. Oh, no doubt. I agree. > Just raid on a lower layer and btrfs on top. That has its own set of problems. The conventional RAID solution on Linux (MD) also has some pretty terrifying corruption edge cases with RAID-5 and RAID-6 (as I explained in [1]) which will bite you if yo…

Won't silent corruption on the raid level be detected by the integrity checks in btrfs? It won't be able to automatically repair it, but it should give errors at least, right?

Re: Bcachefs Goes to "Externally Maintained"

#390

Post by aaronmdjones »

procaryote wrote: post #389
aaronmdjones wrote: post #385

Earlier quoted context omitted.

> If you ignore explicit warnings at mkfs time and then get upset the warning was accurate, you can't really fully blame the file system for it. Oh, no doubt. I agree. > Just raid on a lower layer and btrfs on top. That has its own set of problems. The conventional RAID solution on Linux (MD) also has some pretty terrifying corruption edge cases with RAID-5 and RAID-6 (as I explained in [1]) which will bite you if yo…

Won't silent corruption on the raid level be detected by the integrity checks in btrfs? It won't be able to automatically repair it, but it should give errors at least, right?

Yeah, that would be the "error detection at a higher level" (than MD) part. It'd still be on you to pull one drive at a time from the array until the errors go away (then you know which drive has the corrupted block in that stripe, and can remove the mdadm metadata from it and then re-add it to the array so that the kernel forces a clean resync, reconstructing the good block from the parity). Doing the "repair" action in MD would instead rewrite your good parity for now-corrupted data and you would have no means of recovering. MD can't know whether the data is bad or the parity is bad because it doesn't know what the data is supposed to look like; even if btrfs does have a checksum for it, that's on a higher, disconnected layer. All filesystems on top of a parity MD suffer from this same vulnerability; some of them won't even be able to tell you when a file has become corrupted (e.g. FAT32), leading to this corruption being persisted into backups.
Post reply on HN