Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Btrfs in Linux 6.2 brings performance improvements, better RAID 5/6 reliability

phoronix.com

Post reply on HN
Print view View original

181–190 of 231 posts

Re: Btrfs in Linux 6.2 brings performance improvements, better RAID 5/6 reliability

#181

Post by exceptione »

xorcist wrote: post #177
dale_glass wrote: post #19

But is the RAID handling remotely sane yet? See https://arstechnica.com/gadgets/2021/09/examining-btrfs-linu... It has gems such as: * It won't boot on a degraded array by default, requiring manual action to mount it * It won't complain if one of the disks is stale * It won't resilver automatically if a disk is re-added to the array I think the first is the killer. RAID is a High Availability measure. Your system is…

Remember that DM is still there for you even if you use btrfs. Every discussion about btrfs immediately escalates to how the btrfs RAID support develops. But btrfs is useful on its own without that mode. Should you wish to put a btrfs file system on software RAID (DM), you can still do that. The btrfs RAID modes are something else, where you cut the DM driver out of the picture. That's not going to be as well tested…

DM = device mapper

wikipedia: The device mapper is a framework provided by the Linux kernel for mapping physical block devices onto higher-level virtual block devices. It forms the foundation of the logical volume manager (LVM), software RAIDs and dm-crypt disk encryption, and offers additional features such as file system snapshots.[1]

Device mapper works by passing data from a virtual block device, which is provided by the device mapper itself, to another block device. Data can be also modified in transition, which is performed, for example, in the case of device mapper providing disk encryption or simulation of unreliable hardware behavior.

Re: Btrfs in Linux 6.2 brings performance improvements, better RAID 5/6 reliability

#182

Post by vetinari »

wokkel wrote: post #176
happymellon wrote: post #166

Earlier quoted context omitted.

I've had more issues with my ZFS arrays than the BTRFS ones. The Linux implementation is pretty awful, with it demanding that it uses the absolute `/dev/sdx` reference even if you try to build it using serials or other unchaging reference. Replace a disk and then after the next restart it fails to bring up the RAID because `sdf` points to a different disk. At least BTRFS uses internal UUIDs so can bring up the arrays…

Uh, using /dev/disk/by-id to assemble a zfs array is pretty standard for zol. I never use device names and shuffling disks has never required any manual intervention on my part. Was this a very old ZoL?

At a time, it was the default, and you had to go out of your way to use /dev/disk/by-id (you had to create your pool, export it, and then import it by id), only to be hit by another issue, where when preparing grub, it was looking for the IDs straight in /dev (that's how I learned that ZPOOL_VDEV_NAME_PATH=YES exists).

Yeah, putting / on zfs was not smart. Nowadays, when the distro doesn't support it out of the box and doesn't come with kernel+bootloader+zfs tested together (i.e. proxmox, ubuntu), I wouldn't do it.

Re: Btrfs in Linux 6.2 brings performance improvements, better RAID 5/6 reliability

#184

Post by AshamedCaptain »

boardwaalk wrote: post #141

As someone who is in the middle of pulling apart a BTRFS volume by hand (read: writing code to interpret the data structures) to try and recover it, I think being burnt enough is once. No indication of any hardware issue: No recent power loss (& it's on a UPS), no SMART issues, no memory test positives. But the block tree (at least, WIP) is f*cked across all the disks (looks like two competing writers went at it) and…

This just can't happen with btrfs since the _old_ block tree should be still in the disk somewhere. i.e. to corrupt the image so that btrfs shits itself is easy to do, but the _previous_ version of the data is still there by construction and if you are already manipulating the data structures it should be easy enough to just point the sb to it.

The worst issue I've ever had with btrfs simply required zeroing the journal (by hand -- both the kernel and *fsck tools would crash when reading it). The first time I reported the issue to the mailing list and the underlying issue was promptly fixed. However, it still happened a second time, and I didn't bother reporting it. As many people say on this thread, once is too many when it comes to filesystems.

Re: Btrfs in Linux 6.2 brings performance improvements, better RAID 5/6 reliability

#185

Post by ndsipa_pomu »

dekhn wrote: post #27
dale_glass wrote: post #19

But is the RAID handling remotely sane yet? See https://arstechnica.com/gadgets/2021/09/examining-btrfs-linu... It has gems such as: * It won't boot on a degraded array by default, requiring manual action to mount it * It won't complain if one of the disks is stale * It won't resilver automatically if a disk is re-added to the array I think the first is the killer. RAID is a High Availability measure. Your system is…

Many folks I know who manage storage don't make the boot volume RAID (redundant)- instead, it's some rapidly duplicatable thing like an NMVE flash containing the root filesystem, and there's a replacement handy. Then you can bring up and bring the full power of userspace to bear on the RAID repair.

That's fair enough for a system that isn't critical, but you can almost guarantee that the disk failure will happen at the most inconvenient time. I'd rather have a couple of cheap disks and MD software RAID so that the system boots as long as one of the drives hasn't failed. Then you can replace/fix the failing disk at a convenient time.

Re: Btrfs in Linux 6.2 brings performance improvements, better RAID 5/6 reliability

#186

Post by vetinari »

helf wrote: post #111
Teknoman117 wrote: post #109

Earlier quoted context omitted.

To be fair, it didn't break or lose data. Just got really, really slow until I let it finish reclaiming space.

Yeah. On my end it just slowed to a completely unusable level. So while it technically didn’t lose data it effectively did since I was looking at weeks to replicate any data off it. I let it run for a week and it was still crawling when I gave up (4TB drive)

Was it a SMR drive, by a chance? You can make those crawl with any filesystem. I even had one that couldn't finish initial Time Machine backup.

Re: Btrfs in Linux 6.2 brings performance improvements, better RAID 5/6 reliability

#187

Post by ndsipa_pomu »

cmurf wrote: post #155
dale_glass wrote: post #19

But is the RAID handling remotely sane yet? See https://arstechnica.com/gadgets/2021/09/examining-btrfs-linu... It has gems such as: * It won't boot on a degraded array by default, requiring manual action to mount it * It won't complain if one of the disks is stale * It won't resilver automatically if a disk is re-added to the array I think the first is the killer. RAID is a High Availability measure. Your system is…

Automatic degraded mount at boot time isn't implemented by md either, it's implemented by a dracut module (or whatever creates the initramfs). The gist is that when mdadm assembly fails, the dracut module's script performs a 5 minute wait to see if all devices will appear, and then if not, tries to assemble degraded. These days, with UEFI prevalent, a system won't boot without EFI system partitions replicated (and po…

You can boot from a MD RAID1 system that's degraded easily enough. Here's some info on how to implement it https://outflux.net/blog/archives/2018/04/19/uefi-booting-an...

Re: Btrfs in Linux 6.2 brings performance improvements, better RAID 5/6 reliability

#188

Post by vetinari »

ksec wrote: post #172
imhoguy wrote: post #10

Earlier quoted context omitted.

Unlikely, I am on DSM 7.1 and it is 4.4.180+, although I know heavily patched. I have read DSM 7.2 will land 5.10.

I don't think Synology ever update their kernel with DSM. So the kernel you get when you buy it will be the one that last for its life time. But I do hope there will be model, especially entry model that comes with 5.10. Synology's Linux timeline is simply too slow.

They might update their kernel, just not the kernel version. They seem to be using SoCs in their devices and then respective BSPs by the SoC vendor, which are notorious for not being updated for a new kernel.

I have one of the infamous Atom C2538-based Synology there, and it is running kernel 3.10.108, built in october 2022. The device is 2019 model (the SoC was introduced in 2013).

Re: Btrfs in Linux 6.2 brings performance improvements, better RAID 5/6 reliability

#189

Post by matja »

chasil wrote: post #57
WastingMyTime89 wrote: post #51

Earlier quoted context omitted.

Enlighten me on how a filesystem with an incompatible license is ever going to be mainlined. Having to use an out-of-tree patchset is an immediate no as far as I’m concerned. I also have no intention of switching to an hobbyist operating system, thank you.

I recently tried out the Ubuntu native ZFS installation. I was surprised to see that it taints the kernel when I checked the demsg.

Why is that surprising? It is a difference license. So is the Nvidia driver.

Re: Btrfs in Linux 6.2 brings performance improvements, better RAID 5/6 reliability

#190

Post by kiririn »

ilyt wrote: post #123
josephcsible wrote: post #73

Earlier quoted context omitted.

IMO, the inconvenience this causes is outweighed by the prevention of data loss for people who otherwise would never have noticed the degraded array.

Right but if it's remote server you're fucked and you can't login on it to even assess what happened

That’s what the likes of dropbear-initramfs is for
Post reply on HN