Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Btrfs in Linux 6.2 brings performance improvements, better RAID 5/6 reliability

phoronix.com

Post reply on HN
Print view View original

61–70 of 231 posts

Re: Btrfs in Linux 6.2 brings performance improvements, better RAID 5/6 reliability

#61

Post by [deleted] »

WastingMyTime89 wrote: post #41
herpderperator wrote: post #12

I've been managing a raid6 ext4 array with mdadm for 10 years. Started with 4 x 4TB disks and kept adding, up to 11 disks now. It works reliably and as designed. Had a few disk failures and replaced them without issues. That's one of the nice things about mdadm vs ZFS: you can add and remove disks from the array as you see fit, rather than being forced to upgrade all disks if you want to increase the size of your arr…

Why is this comment downvoted? ZFS limited disk management forcing you to use the size of the smaller disk and making it difficult to resize array is not a feature. It’s a serious limitation when using it as a normal user and not in a professional environment. It’s extremely annoying that most of the ZFS zealots are in denial.

[deleted]

Re: Btrfs in Linux 6.2 brings performance improvements, better RAID 5/6 reliability

#62

Post by _kdave »

seanw444 wrote: post #9

Glad to see Btrfs getting continual updates. It's my favorite filesystem for my personal machines (work and home PCs). The feature set is just awesome. I just hope it doesn't get completely abandoned as its development seems to have slowed significantly. The only thing it's missing before I consider it full-featured is stable RAID 5/6. But it looks like that hasn't been forgotten.

The development speed has to balance the schedule of linux kernel development (merge window, release candidates, 3 months cycle) and demand to merge several distinct features or core changes. There are no formal deadlines but we have to make sure that the new code is feasible to be stabilized in the given time. Once a new feature is in the wild some bugs or fixups are still needed so this takes some time from the new development and has to be accounted for.

My strategy to pull new things is to have one big feature that has ideally been reviewed and iterated in the mailinglist or there was a lot of testing already done. In addition two smaller features can be merged, with limited scope, not affecting default setup and possibly easy to debug/fix/revert if needed. Besides that there are cleanups or core updates going on so this should not touch the same code to make testing less painful. With new features the test matrix grows, code might need wide cleanups or generalizaionts before the actual feature code is merged. So this can indeed slow down development.

The raid56 is progressing but until the 6.2 pull from today there was not much to announce regarding stability/reliability. There were proposed fixes but as incompatible features, which means some changes on the user side and with backward compatibility issues. What's pending for 6.2 should fix one of the bad problems at least for raid5.

Re: Btrfs in Linux 6.2 brings performance improvements, better RAID 5/6 reliability

#63

Post by viraptor »

dale_glass wrote: post #56
chomp wrote: post #47

Earlier quoted context omitted.

Why? Are you in a position where you can’t modify grub or fstab?

Because RAID is a High Availability technology. It's not a backup. It doesn't protect against mistakes, bugs, or the server failing in some other way. What it's good for is for ensuring that work keeps happening if a disk fails. And disk failures happen to be fairly frequent, since spinning rust is a rather delicate technology. If somebody has to connect to the machine and fiddle with it by hand it means that the sys…

You don't have to modify it after a failure. You can set it right now to explicitly say "when I end up in a degraded state, I want to boot anyway".

> What it's good for is for ensuring that work keeps happening if a disk fails.

Different services have different priorities. I want my work servers to keep running, but my home server under the desk to stop until I replace the failed drive. Btrfs gives you a choice.

Re: Btrfs in Linux 6.2 brings performance improvements, better RAID 5/6 reliability

#64

Post by viraptor »

doublepg23 wrote: post #54
viraptor wrote: post #14

Earlier quoted context omitted.

Ideally we'd be working based on actual information rather than global mindshare. There will be guinea pigs to test it. There are already quite a few people running it despite the warnings. As much as I want to see a wide use of bcachefs, it's still years away. As someone who actually wants to store data - why would you direct resources to bcachefs which is known experimental, rather than btrfs which plainly document…

Is bcachefs years away? My understanding is the critical chain of code is pretty well tested - bcache.

From widespread use, yes. Bcache was already stable at least 7 years ago. Bcachefs it's not even included in upstream kernel yet, much less any distro as an official option. There's only some plan to start submitting it soon™ and it will take months to complete.

Re: Btrfs in Linux 6.2 brings performance improvements, better RAID 5/6 reliability

#65

Post by nix23 »

formerly_proven wrote: post #6
nix23 wrote: post #3

Oh yeah, i like reliable filesystems like xfs or zfs ;)

extraneous files seroed Though I've used XFS a lot over the years. Mostly because the Debian installer gave 12yo me the choice between ext2, ext3 and xfs, so XFS it was because it sounded cooler.

>so XFS it was because it sounded cooler.

That's what redhat though ;)

Re: Btrfs in Linux 6.2 brings performance improvements, better RAID 5/6 reliability

#66

Post by herpderperator »

oarsinsync wrote: post #29
herpderperator wrote: post #12

I've been managing a raid6 ext4 array with mdadm for 10 years. Started with 4 x 4TB disks and kept adding, up to 11 disks now. It works reliably and as designed. Had a few disk failures and replaced them without issues. That's one of the nice things about mdadm vs ZFS: you can add and remove disks from the array as you see fit, rather than being forced to upgrade all disks if you want to increase the size of your arr…

I did that for a long time. I reached 15 disks. Then something went wrong, and I lost everything. I now run multiple 6 disk raidz2s, in part because I wanted to remove the option that led me down the bad path. I hope your journey ends better than mine did.

Sorry that you lost everything. Keep in mind that RAID is not backup. I also don't understand the correlation between reaching a certain number of disks vs any other scenario. Disaster can always strike, so it's best to be prepared.

Re: Btrfs in Linux 6.2 brings performance improvements, better RAID 5/6 reliability

#67

Post by Hello71 »

_kdave wrote: post #36
dale_glass wrote: post #19

But is the RAID handling remotely sane yet? See https://arstechnica.com/gadgets/2021/09/examining-btrfs-linu... It has gems such as: * It won't boot on a degraded array by default, requiring manual action to mount it * It won't complain if one of the disks is stale * It won't resilver automatically if a disk is re-added to the array I think the first is the killer. RAID is a High Availability measure. Your system is…

> It won't boot on a degraded array by default, requiring manual action to mount it If you want it to behave like that then add 'degraded' to fstab. That a device is missing can have unknown reasons, the user should know better and resolve it or allow such boot. It's not automatic as there's no way to inform the user that it's degraded state.

according to https://arstechnica.com/gadgets/2021/09/examining-btrfs-linu..., btrfs devs say you should not add degraded to fstab, and doing so can easily result in data loss.

Re: Btrfs in Linux 6.2 brings performance improvements, better RAID 5/6 reliability

#69

Post by pedrocr »

walrus01 wrote: post #5

I wonder how this compares to just using mdadm block device level raid5 or raid6. And then a normal filesystem on top.

In theory it should allow fine grained control over raid levels at the file/directory/subvolume level. So that depending on the specific needs you can set direct/raid1/raid10/raid5/raid6 without having to setup different arrays for each level you want to use and then have to manage the relative space between them. I can imagine uses for that but the reliability of the more tested solutions is much more important.

Re: Btrfs in Linux 6.2 brings performance improvements, better RAID 5/6 reliability

#70

Post by rtp4me »

sliken wrote: post #34
warmwaffles wrote: post #15

Earlier quoted context omitted.

The flexibility of software RAID is nice that you can mix and match hard drive manufacturers and generally have zero issues. For hardware RAID, I've always been told to stick to one drive family from one manufacturer and not to mix and match.

Generally every recommendation I've heard is don't use hardware RAID. Some of the problems: * RAID cards usually have under powered CPUs, and can easily be a bottleneck. Often the best performance with a RAID card is with the RAID disabled. * Even battery backed RAM is often pretty slow and on the wrong end of a high latency connection between CPU/RAM and disks. Generally investments in ram or intentlog/writelog/slog…

My experience with hardware RAID cards (LSI, PERC, etc) is vastly different than the recommendations you heard.

I have been running production servers with LSI hardware RAID cards for the past 15 years and have not experienced the items you note. The only thing I did experience was a bad RAID card (solved by replacing it with a similar LSI model).

To contrast your "problem" list:

  * Compared with MDADM (mirrors, RAID-5, RAID-6), the LSI RAID cards tend to be on par as far as performance (esp with BBU)
  * A failed drive won't prevent the array from coming on line (in my experience)
  * All my servers have battery backed RAM; never had an issue with slowness or high-latency.  Can't say the same for ZFS.
  * Never, ever had a problem with metadata on the disk.  Not sure why this seems to be an issue
  * LSI cards have pass-thru mode and can easily be used with BTRFS/ZFS.  I have done lots of tests; never an issue
  * LSI cards have their own "patrol read" mechansim to scan drives for bad sectors and send alerts when something seems wrong
On a positive note:

  * Hard drive failures are truly plug-n-play.  Send a tech out to the cabinet, spot the RED light, replace drive.  Done.  No OS work, no console, no crash cart needed.  
  * You can logically divide the drives using the RAID manager tools just like Linux MDADM, ZFS, BTRFS, etc.  Easy.
  * You don't need the specific card to replace a failed card.  You just need a card that can read the metadata on the array to bring it back to life.
  * Lots of monitoring scripts available in the wild to get RAID stats, rebuild times, etc.
I am not saying hardware cards are indestructible. But, all in all, hardware RAID cards are solid devices that have been in many, many production servers all over the world. I certainly would not dismiss them in favor of ZFS RAID (which tends to be slow for many operations). Pick the best tool for the job.
Post reply on HN