Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Btrfs in Linux 6.2 brings performance improvements, better RAID 5/6 reliability

phoronix.com

Post reply on HN
Print view View original

81–90 of 231 posts

Re: Btrfs in Linux 6.2 brings performance improvements, better RAID 5/6 reliability

#81

Post by WastingMyTime89 »

skyyler wrote: post #53
WastingMyTime89 wrote: post #51

Earlier quoted context omitted.

Enlighten me on how a filesystem with an incompatible license is ever going to be mainlined. Having to use an out-of-tree patchset is an immediate no as far as I’m concerned. I also have no intention of switching to an hobbyist operating system, thank you.

You're already using a hobbyist operating system... Your horse isn't as high as you think.

Linux hasn’t really been a hobbyist operating system for years as anyone taking even a casual glance at a list of contributors would know but sure let’s all pretend FreeBSD is somehow relevant or properly maintained. That’s probably in the same universe where ZFS has a chance of being mainlined and isn’t heavily encumbered by Oracle owned patents.

Why people would contribute to this mess is beyond me but everyone is free to do what they want.

Re: Btrfs in Linux 6.2 brings performance improvements, better RAID 5/6 reliability

#82

Post by simcop2387 »

DominoTree wrote: post #35

I've never lost data with btrfs, but I have purposefully avoided anything other than basic disk configurations. I use it on my workstations because it's the default in Fedora these days and haven't had any complaints. On the flip-side, I've done some of the most horrible things possible and screwed up my 30-disk ZFS array a number of times and I've never lost data. I doubt btrfs could recover from anything I've done…

Only time I've lost data with zfs was when i was first doing something supremely stupid, mostly as a dare to see if it would work.

I had recently upgraded my pool and had a bunch of old disks lying around so i fired them up in a new pool, 6x 4TB and 6x 8TB disks. I took the 4TB disks and set them in in a raid0 to get effectively 9x 8TB disks. Apparently doing this will work but you MUST make sure you properly wipe all of the newly raided disks of their old ZFS data or you can end up with ZFS trying to check that pool and breaking the mdadm superblocks.

After I properly zeroed the 4TB disks it's been working fine for weeks. This let me set it up a pool in raidz3 with them all so i can loose any of the much older 4tb disks or up to 3 of the 8tb ones. I don't use it for anything that's absolutely critical to store, just random bulk data that i don't want to worry that much about, but it's been reliable enough since then.

Re: Btrfs in Linux 6.2 brings performance improvements, better RAID 5/6 reliability

#83

Post by yjftsjthsd-h »

WastingMyTime89 wrote: post #37
candiddevmike wrote: post #8

Maybe I'm just cynical but I think the ship has sailed for BTRFS RAID 5/6. It's now part of the global mindshare that BTRFS RAID 5/6 == data loss, no one wants to be the guinea pig that proves it works. Better to direct resources towards bcachefs or ZFS IMO.

ZFS will never be mainlined so it’s beyond useless.

DKMS is fine, and Ubuntu even ships prebuilt .ko files; it works fine in practice.

Re: Btrfs in Linux 6.2 brings performance improvements, better RAID 5/6 reliability

#84

Post by yjftsjthsd-h »

WastingMyTime89 wrote: post #81
skyyler wrote: post #53

Earlier quoted context omitted.

You're already using a hobbyist operating system... Your horse isn't as high as you think.

Linux hasn’t really been a hobbyist operating system for years as anyone taking even a casual glance at a list of contributors would know but sure let’s all pretend FreeBSD is somehow relevant or properly maintained. That’s probably in the same universe where ZFS has a chance of being mainlined and isn’t heavily encumbered by Oracle owned patents. Why people would contribute to this mess is beyond me but everyone is…

> Linux hasn’t really been a hobbyist operating system for years as anyone taking even a casual glance at a list of contributors would know but sure let’s all pretend FreeBSD is somehow relevant or properly maintained.

You haven't glanced at a list of FreeBSD's contributors, have you?

Re: Btrfs in Linux 6.2 brings performance improvements, better RAID 5/6 reliability

#85

Post by blibble »

WastingMyTime89 wrote: post #37
candiddevmike wrote: post #8

Maybe I'm just cynical but I think the ship has sailed for BTRFS RAID 5/6. It's now part of the global mindshare that BTRFS RAID 5/6 == data loss, no one wants to be the guinea pig that proves it works. Better to direct resources towards bcachefs or ZFS IMO.

ZFS will never be mainlined so it’s beyond useless.

I'll dump Linux before I dump ZFS

Re: Btrfs in Linux 6.2 brings performance improvements, better RAID 5/6 reliability

#86

Post by antod »

dekhn wrote: post #27
dale_glass wrote: post #19

But is the RAID handling remotely sane yet? See https://arstechnica.com/gadgets/2021/09/examining-btrfs-linu... It has gems such as: * It won't boot on a degraded array by default, requiring manual action to mount it * It won't complain if one of the disks is stale * It won't resilver automatically if a disk is re-added to the array I think the first is the killer. RAID is a High Availability measure. Your system is…

Many folks I know who manage storage don't make the boot volume RAID (redundant)- instead, it's some rapidly duplicatable thing like an NMVE flash containing the root filesystem, and there's a replacement handy. Then you can bring up and bring the full power of userspace to bear on the RAID repair.

The way I used to do it (back then getting grub/initrd raid aware wasn't really doable - not sure about now) with mdadm was to use RAID1 on the boot volume such that a disk from a broken mirror was still usable by itself. Other volumes were RAID 5 or 10 etc.

Re: Btrfs in Linux 6.2 brings performance improvements, better RAID 5/6 reliability

#87

Post by sliken »

rtp4me wrote: post #70
sliken wrote: post #34

Earlier quoted context omitted.

Generally every recommendation I've heard is don't use hardware RAID. Some of the problems: * RAID cards usually have under powered CPUs, and can easily be a bottleneck. Often the best performance with a RAID card is with the RAID disabled. * Even battery backed RAM is often pretty slow and on the wrong end of a high latency connection between CPU/RAM and disks. Generally investments in ram or intentlog/writelog/slog…

My experience with hardware RAID cards (LSI, PERC, etc) is vastly different than the recommendations you heard. I have been running production servers with LSI hardware RAID cards for the past 15 years and have not experienced the items you note. The only thing I did experience was a bad RAID card (solved by replacing it with a similar LSI model). To contrast your "problem" list: * Compared with MDADM (mirrors, RAID-…

Glad it's worked for you. I don't think I've had anything that conflicts with your experiences. I didn't mean to imply a degraded hardware RAID wouldn't mount, I think that's a BTRFS RAID5 issue. Generally I'd consider BTRFS a toy, doubly so for RAID5/6 on BTRFS.

Generally I (and colleagues) consider a hardware RAID failure a nightmare and a SAS HBA failure an annoyance. Doubly so if your storage design included cross connected servers and you just mount the storage from the other server. I've never tried similar with a hardware RAID. Can you cross mount and easily import/export RAID sets between controllers?

The main hardware RAID performance issues I see is with higher drive counts or NVME drives when combined with RAID6. Seen any hardware RAIDs and can manage a few GB/sec with 2 disks of redundancy? 3 disks? Even spending a few $k on hardware RAID seems to lose to a random 5 year old server using ZFS or software RAID by a large factor. Seems like even pretty old x86-64 servers manage a few GB/sec per core.

Careful on the passthru, I had many generations of LSI cards that worked, and 3ware and Areca before that. However I've been hearing that the new LSI RAID cards lack passthru/JBOD mode. I've seen reviews and complaints about the lack of it. One case had involved a LSI hardware RAID connected to a Dell JBOD array, but maybe it was specifically crippled by Dell.

I avoided ZFS for years, generally slower at the relevant workloads (used IO logs collected with systemtap and representative loads created with FIO). However once I added cache and the benchmark included multiple streams of writes, ZFS was a huge win. In particular 64 or more sequential write streams to 120 disks. In fact ZFS did better with 3 disks of redundancy that other filesystems did with 2.

In that past I had more luck with ZFS or MDADM for doing things like /dev/sd[ab]1 for a 32gb RAID1 for boot, then /dev/sd[abcd]2 for RAID5 or RAIDz2. Sounds like the hardware RAID tools are get better.

I've heard of some hardware RAID setups managing to recover hardware RAID metadata with a new card, and some not. Even cases of successes and failures from the same company. Even cases where support claimed it would work, but then decided the card and/or firmware wasn't close enough. Sure decent support can overnight a card, but nowhere near as nice as just being able to mount the drives on any Linux box.

Re: Btrfs in Linux 6.2 brings performance improvements, better RAID 5/6 reliability

#88

Post by bravetraveler »

dekhn wrote: post #27
dale_glass wrote: post #19

But is the RAID handling remotely sane yet? See https://arstechnica.com/gadgets/2021/09/examining-btrfs-linu... It has gems such as: * It won't boot on a degraded array by default, requiring manual action to mount it * It won't complain if one of the disks is stale * It won't resilver automatically if a disk is re-added to the array I think the first is the killer. RAID is a High Availability measure. Your system is…

Many folks I know who manage storage don't make the boot volume RAID (redundant)- instead, it's some rapidly duplicatable thing like an NMVE flash containing the root filesystem, and there's a replacement handy. Then you can bring up and bring the full power of userspace to bear on the RAID repair.

Plain old USB thumb drives are pretty common for this

I can't knock it, but I imagine I'd be yelling for at least RAID1 if I had to wait for a replacement.

Re: Btrfs in Linux 6.2 brings performance improvements, better RAID 5/6 reliability

#89

Post by eikenberry »

candiddevmike wrote: post #8

Maybe I'm just cynical but I think the ship has sailed for BTRFS RAID 5/6. It's now part of the global mindshare that BTRFS RAID 5/6 == data loss, no one wants to be the guinea pig that proves it works. Better to direct resources towards bcachefs or ZFS IMO.

Ideally on a ZFS replacement that has a compatible license.

Re: Btrfs in Linux 6.2 brings performance improvements, better RAID 5/6 reliability

#90

Post by Matthias247 »

Gigachad wrote: post #43
DominoTree wrote: post #35

I've never lost data with btrfs, but I have purposefully avoided anything other than basic disk configurations. I use it on my workstations because it's the default in Fedora these days and haven't had any complaints. On the flip-side, I've done some of the most horrible things possible and screwed up my 30-disk ZFS array a number of times and I've never lost data. I doubt btrfs could recover from anything I've done…

I feel like the data loss risk doesn't really make sense to be worried about. It doesn't matter what FS or storage system you use, you _need_ a backup if you actually care about the data. And if you have backups, you are fine. As a random anecdote I've been using BTRFS for everything for over 7 years now and had no issues. Think the only FS I've ever had problems with was ExFAT.

data corruption is slightly orthogonal to data loss.

If your filesystem silently corrupts some data in a way you don't notice, you might just replicate the corrupted data to your backup system over time. And in that case you will still have lost it.

Data loss - e.g. due to a disk failure - is a lot more obvious.

Post reply on HN