Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Btrfs in Linux 6.2 brings performance improvements, better RAID 5/6 reliability

phoronix.com

Post reply on HN
Print view View original

151–160 of 231 posts

Re: Btrfs in Linux 6.2 brings performance improvements, better RAID 5/6 reliability

#151

Post by justsomehnguy »

cycomanic wrote: post #118
klysm wrote: post #24

Earlier quoted context omitted.

> * It won't boot on a degraded array by default, requiring manual action to mount it That by itself is a complete deal breaker.

Hold on, people make this out to be like if a disk fails it requires immediate intervention. However if the system already booted it stays up. Who reboots a production system without monitoring it? Also if you boot with a degraded disk are you not asking for massive trouble, because if another disk fails you might end up with data loss which is IMO much worse on a production system than not being able to boot until y…

> Who reboots a production system without monitoring it?

Anyone.

> Also if you boot with a degraded disk are you not asking for massive trouble

> not being able to boot until you add another disk

Great.

I'm just a [sys]admin who were given a task to do something on $server.

I jump around the red tape, claw out a 15 minute downtime because the task requires reboot, proceed with all that corporate dance with notification emails.

I do my thing, reboot the server and it doesn't return back online.

Suddenly I broke the server, missed maintenance window, amount of mails with CC and RE: in my mailbox grows in geometric progression and the most important - now I need to find out who were responsible for the server, contact him and [kick his ass] ask him to diagnose what is going on.

Bonus points if:

server doesn't have a meaningful BMC/iLO/iDRAC with KVM console

it does have it but it's broken for some reason eg requires Java 6 on Vista

server was configured 10 years ago by a greybeard who is not only retired but already died from the old age

server is 6000km away from any place with a replacement disks and the earliest time when you can send the replacement would be the next spring when the ice would break and taw enough for the ships to move. Of course you can hire a helo to deliver it, which get your ass chewed for an unplanned $50k expenses

Also: https://news.ycombinator.com/item?id=33963000

Re: Btrfs in Linux 6.2 brings performance improvements, better RAID 5/6 reliability

#152

Post by encryptluks2 »

freeqaz wrote: post #80

We run BTRFS on our work laptops and have been for a few years now. Keep in mind that we're only 4 devs, so our sample size is small, but we haven't had any issues with it so far and it's been very pleasant to use so far! We published our internal doc for how we install our Arch setup with fully encrypted BTRFS if anybody is curious. Happy to answer any questions too! https://www.lunasec.io/docs/blog/arch-linux-insta…

That is a pretty awesome article. I recently reinstalled Arch on a new PC and used archinstall to setup encryption, but they had a few bad defaults that I had to later correct... like using a sha512 hash algo, which slowed down opening the drives significantly, and somehow adding 2 different types of compression to the fstab. Once that was fixed, it has been working great.

You have a typo in your article BTW:

> aes-exts-plain64

Re: Btrfs in Linux 6.2 brings performance improvements, better RAID 5/6 reliability

#153

Post by wang_li »

cycomanic wrote: post #118
klysm wrote: post #24

Earlier quoted context omitted.

> * It won't boot on a degraded array by default, requiring manual action to mount it That by itself is a complete deal breaker.

Hold on, people make this out to be like if a disk fails it requires immediate intervention. However if the system already booted it stays up. Who reboots a production system without monitoring it? Also if you boot with a degraded disk are you not asking for massive trouble, because if another disk fails you might end up with data loss which is IMO much worse on a production system than not being able to boot until y…

The risks are the same whether you keep running with a failed disk vs. rebooting/booting with a failed disk. Also, given a system in which the OS is not running and requires booting, how exactly are you going to resync a replacement drive if you don't boot the system with a degraded raid volume?

Re: Btrfs in Linux 6.2 brings performance improvements, better RAID 5/6 reliability

#154

Post by cmurf »

chasil wrote: post #49
dale_glass wrote: post #19

But is the RAID handling remotely sane yet? See https://arstechnica.com/gadgets/2021/09/examining-btrfs-linu... It has gems such as: * It won't boot on a degraded array by default, requiring manual action to mount it * It won't complain if one of the disks is stale * It won't resilver automatically if a disk is re-added to the array I think the first is the killer. RAID is a High Availability measure. Your system is…

Regarding resilvering, I am amazed by ZFS' capability of quickly bringing a stale mirror back into synchronization. I have used the FUSE port of ZFS to write only one member of a mirroset, then upon mounting both members elsewhere, the stale mirror was very quickly resilvered, so ZFS was able to determine only the blocks needing to be refreshed. In btrfs, I understand that this requires a rebalance, which will read/w…

Scrub, not balance. It's primarily a read operation. The on-disk format has all the information needed to do an automatic abbreviated scrub, but the feature isn't implemented yet.

Re: Btrfs in Linux 6.2 brings performance improvements, better RAID 5/6 reliability

#155

Post by cmurf »

dale_glass wrote: post #19

But is the RAID handling remotely sane yet? See https://arstechnica.com/gadgets/2021/09/examining-btrfs-linu... It has gems such as: * It won't boot on a degraded array by default, requiring manual action to mount it * It won't complain if one of the disks is stale * It won't resilver automatically if a disk is re-added to the array I think the first is the killer. RAID is a High Availability measure. Your system is…

Automatic degraded mount at boot time isn't implemented by md either, it's implemented by a dracut module (or whatever creates the initramfs). The gist is that when mdadm assembly fails, the dracut module's script performs a 5 minute wait to see if all devices will appear, and then if not, tries to assemble degraded.

These days, with UEFI prevalent, a system won't boot without EFI system partitions replicated (and possibly sync'd) on every drive.

Re: Btrfs in Linux 6.2 brings performance improvements, better RAID 5/6 reliability

#156

Post by wtallis »

justsomehnguy wrote: post #146
p1necone wrote: post #107

Earlier quoted context omitted.

> That's the only purpose for it. That's not the only purpose for it. There's three reasons I can think of that you might set up a RAID array: * You want better uptime. (your use case) * You want to protect from data loss. (my assumption was that this is the most common use case, but I could be wrong. This also helps with uptime because there's nothing worse for uptime than having to restore lost data from a cold bac…

> Booting a RAID array with a failed disk is a bad idea Booting a RAID array with a failed disk is absolutely fine idea. How else I get access to the tools to identify the bad drive and resilver RAID on a replacement, be it in the same bay or not?

Booting from a degraded array is only a fine idea in some circumstances, not all. That's why the kernel should not default to automatically doing so; but a distro or sysadmin that has better knowledge of the broader situation (eg. presence of hot spares or a working monitoring/alert system) can reasonably change that default when the risks of booting from a degraded array have been mitigated.

Re: Btrfs in Linux 6.2 brings performance improvements, better RAID 5/6 reliability

#157

Post by cmurf »

_kdave wrote: post #36
dale_glass wrote: post #19

But is the RAID handling remotely sane yet? See https://arstechnica.com/gadgets/2021/09/examining-btrfs-linu... It has gems such as: * It won't boot on a degraded array by default, requiring manual action to mount it * It won't complain if one of the disks is stale * It won't resilver automatically if a disk is re-added to the array I think the first is the killer. RAID is a High Availability measure. Your system is…

> It won't boot on a degraded array by default, requiring manual action to mount it If you want it to behave like that then add 'degraded' to fstab. That a device is missing can have unknown reasons, the user should know better and resolve it or allow such boot. It's not automatic as there's no way to inform the user that it's degraded state.

Most distros have a udev rule in place that inhibits a mount attempt of a multiple device Btrfs until all devices are visible to the kernel. The degraded mount option won't even matter in this case, because mount isn't attempted.

If you remove this udev rule and then add degraded mount option to fstab, it's very risky because now even a small delay in drives appearing can result in a degraded mount. And it's even possible to get a split brain situation.

Btrfs needs automatic abbreviated scrub, akin to the mdadm write intent bitmap which significantly reduces the resync operation.

Re: Btrfs in Linux 6.2 brings performance improvements, better RAID 5/6 reliability

#158

Post by justsomehnguy »

wtallis wrote: post #156
justsomehnguy wrote: post #146

Earlier quoted context omitted.

> Booting a RAID array with a failed disk is a bad idea Booting a RAID array with a failed disk is absolutely fine idea. How else I get access to the tools to identify the bad drive and resilver RAID on a replacement, be it in the same bay or not?

Booting from a degraded array is only a fine idea in some circumstances, not all. That's why the kernel should not default to automatically doing so; but a distro or sysadmin that has better knowledge of the broader situation (eg. presence of hot spares or a working monitoring/alert system) can reasonably change that default when the risks of booting from a degraded array have been mitigated.

Ie you are treating RAID as a backup.

Re: Btrfs in Linux 6.2 brings performance improvements, better RAID 5/6 reliability

#159

Post by wtallis »

justsomehnguy wrote: post #151
cycomanic wrote: post #118

Earlier quoted context omitted.

Hold on, people make this out to be like if a disk fails it requires immediate intervention. However if the system already booted it stays up. Who reboots a production system without monitoring it? Also if you boot with a degraded disk are you not asking for massive trouble, because if another disk fails you might end up with data loss which is IMO much worse on a production system than not being able to boot until y…

> Who reboots a production system without monitoring it? Anyone. > Also if you boot with a degraded disk are you not asking for massive trouble > not being able to boot until you add another disk Great. I'm just a [sys]admin who were given a task to do something on $server. I jump around the red tape, claw out a 15 minute downtime because the task requires reboot, proceed with all that corporate dance with notificati…

It sounds like you're hypothesizing a long chain of bad decisions, and then ridiculing btrfs for taking the choice that means your next bad decision only culminates in (predictable, preventable) downtime rather than data loss.

Re: Btrfs in Linux 6.2 brings performance improvements, better RAID 5/6 reliability

#160

Post by linuxandrew »

alerighi wrote: post #91
dekhn wrote: post #27

Earlier quoted context omitted.

Many folks I know who manage storage don't make the boot volume RAID (redundant)- instead, it's some rapidly duplicatable thing like an NMVE flash containing the root filesystem, and there's a replacement handy. Then you can bring up and bring the full power of userspace to bear on the RAID repair.

I don't get why you need to do so. RAID is not a substitute of backups, and shouldn't used that way. Meaning that of course you have a backup of the data that you can recover from. RAID is used for two things: 1. improving the performance of slow disks (at least the read performance) 2. having 1/2 disk fail with your system remaining completely usable, till you replace the faulty disk as soon as possible. The second…

3. Prevent bit-rot w/ software RAID. Running ZFS, btrfs et al. on a single disk can detect bit-rot but not repair it.
Post reply on HN