Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Btrfs in Linux 6.2 brings performance improvements, better RAID 5/6 reliability

phoronix.com

Post reply on HN
Print view View original

131–140 of 231 posts

Re: Btrfs in Linux 6.2 brings performance improvements, better RAID 5/6 reliability

#131

Post by wil421 »

zeotroph wrote: post #104
simcop2387 wrote: post #82

Earlier quoted context omitted.

Only time I've lost data with zfs was when i was first doing something supremely stupid, mostly as a dare to see if it would work. I had recently upgraded my pool and had a bunch of old disks lying around so i fired them up in a new pool, 6x 4TB and 6x 8TB disks. I took the 4TB disks and set them in in a raid0 to get effectively 9x 8TB disks. Apparently doing this will work but you MUST make sure you properly wipe al…

Is there a zfs native way to create raid0 disks now? On Linux I usually set up a linear mapping with dmsetup, and then point zfs to that. But this hides the native disks from zfs. I've done that a couple times to make a bunch of mismatched disk fit into zfs's requirement that all are of the same size. The ability to work with a mix of disk sizes is one advantage of btrfs - if you remember to be extra careful on a dis…

You can stripe disks (raid0) in ZFS but you should not be using any hardware raid solution, motherboard or pcie card. Host bus adapters are preferred, anything with direct access to the disks.

Re: Btrfs in Linux 6.2 brings performance improvements, better RAID 5/6 reliability

#132

Post by nwallin »

ludocode wrote: post #116
Tuna-Fish wrote: post #31

Earlier quoted context omitted.

Are you using Raid6 for both data and metadata? If yes, you could just as well have been running raid 0 and having more space and performance out of the same disks. Because of well documented issues, metadata on raid6 is not yet safe in power/hw failure situations. And if you have no safety in those situations, what is raid even for? (It's perfectly safe to run metadata on raid1 and data on raid5/6.)

With data on raid6 you should be running metadata on raid1c3 or raid1c4, not raid1 nor raid5/6.

Wait, what's wrong with metadata on raid1?

Re: Btrfs in Linux 6.2 brings performance improvements, better RAID 5/6 reliability

#133

Post by tagrun »

chasil wrote: post #55
DominoTree wrote: post #35

I've never lost data with btrfs, but I have purposefully avoided anything other than basic disk configurations. I use it on my workstations because it's the default in Fedora these days and haven't had any complaints. On the flip-side, I've done some of the most horrible things possible and screwed up my 30-disk ZFS array a number of times and I've never lost data. I doubt btrfs could recover from anything I've done…

Btrfs is able to do one important thing that ZFS cannot: defragment. I see XFS as the performance leader (appears on TPC.org the most often that I can see), btrfs as the fullest featured, and ZFS with the strongest reliability.

That's incorrect, you can defrag with

    btrfs filesystem defragment /path
There is also the mount option autodefrag

Re: Btrfs in Linux 6.2 brings performance improvements, better RAID 5/6 reliability

#134

Post by wtallis »

p1necone wrote: post #107
dale_glass wrote: post #45

Earlier quoted context omitted.

I don't quite understand the use case here. If I'm setting up RAID it's because I want the system to stay up. That's the only purpose for it. If a device goes missing for "unknown reasons", then the machine should still work, and I'll figure out what happened when monitoring pokes me and says RAID is degraded.

> That's the only purpose for it. That's not the only purpose for it. There's three reasons I can think of that you might set up a RAID array: * You want better uptime. (your use case) * You want to protect from data loss. (my assumption was that this is the most common use case, but I could be wrong. This also helps with uptime because there's nothing worse for uptime than having to restore lost data from a cold bac…

Another way of thinking about it: should uptime with bad data or services making false guarantees about data durability actually count as uptime?

Re: Btrfs in Linux 6.2 brings performance improvements, better RAID 5/6 reliability

#135

Post by wtallis »

nwallin wrote: post #132
ludocode wrote: post #116

Earlier quoted context omitted.

With data on raid6 you should be running metadata on raid1c3 or raid1c4, not raid1 nor raid5/6.

Wait, what's wrong with metadata on raid1?

Metadata on raid1 doesn't guarantee the resilience to two-drive failures that raid6 is used for, but metadata on raid1c3 does give you that degree of resilience (with more space overhead, but that's seldom a problem for the metadata).

Re: Btrfs in Linux 6.2 brings performance improvements, better RAID 5/6 reliability

#136

Post by wtallis »

Scaevolus wrote: post #30
denkmoon wrote: post #21

Earlier quoted context omitted.

Do Meta not develop btrfs for their internal use? I don't think community sentiment is a big factor for them.

Meta and related companies have much better systems available for durability than RAID5. I would be moderately surprised if they commonly use any standard RAID level.

Using traditional RAID (or moderate improvements on it, such as provided by btrfs and zfs) to provide drive-level redundancy seems like a waste of time when you also have to worry about redundancy between servers, racks, datacenters and regions.

All of the arguments for why it's better to have something like zfs handle RAID rather than layering something on top of a traditional hardware RAID controller also work for explaining why you should prefer managing storage more globally rather than layering on top of something like zfs—if you have the resources to develop and maintain a true "full stack" storage solution. Which Facebook/Meta obviously does, when they can do things like publish their own spec documents that SSD vendors design around.

Re: Btrfs in Linux 6.2 brings performance improvements, better RAID 5/6 reliability

#137

Post by viraptor »

ilyt wrote: post #127
viraptor wrote: post #14

Earlier quoted context omitted.

Ideally we'd be working based on actual information rather than global mindshare. There will be guinea pigs to test it. There are already quite a few people running it despite the warnings. As much as I want to see a wide use of bcachefs, it's still years away. As someone who actually wants to store data - why would you direct resources to bcachefs which is known experimental, rather than btrfs which plainly document…

Right but "actual information" here is "developers over and over fail to nail what ZFS and MDRAID did well decades ago". There is no reason to trust their next try will be "the one"

Did well in what context? RAID5? There were enough people with motivation to spend time on it rather than other things for mdadm/zfs. It's not like btrfs devs wouldn't know how to implement it if you paid them to work full time on it.

Re: Btrfs in Linux 6.2 brings performance improvements, better RAID 5/6 reliability

#138

Post by ttfkam »

tagrun wrote: post #133
chasil wrote: post #55

Earlier quoted context omitted.

Btrfs is able to do one important thing that ZFS cannot: defragment. I see XFS as the performance leader (appears on TPC.org the most often that I can see), btrfs as the fullest featured, and ZFS with the strongest reliability.

That's incorrect, you can defrag with btrfs filesystem defragment /path There is also the mount option autodefrag

Reread the comment you replied to. Slowly.

Re: Btrfs in Linux 6.2 brings performance improvements, better RAID 5/6 reliability

#139

Post by WastingMyTime89 »

skyyler wrote: post #97
WastingMyTime89 wrote: post #81

Earlier quoted context omitted.

Linux hasn’t really been a hobbyist operating system for years as anyone taking even a casual glance at a list of contributors would know but sure let’s all pretend FreeBSD is somehow relevant or properly maintained. That’s probably in the same universe where ZFS has a chance of being mainlined and isn’t heavily encumbered by Oracle owned patents. Why people would contribute to this mess is beyond me but everyone is…

You're pretending that FreeBSD isn't relevant... You're probably behind a pfsense firewall right now. You have probably seen someone playing games on a PlayStation 3 or 4. You've definitely benefitted from the FreeBSD project without even realising. Have you taken a casual glance at the list of FreeBSD contributors?

I know FreeBSD is used by Sony on its console, by Netflix and by WhatsApp. Everyone knows because they are pretty much the sole serious contributors to it. I don’t think it’s particularly relevant nor do I think it’s particularly good to be honest but we are getting far from the initial discussion about ZFS.

Re: Btrfs in Linux 6.2 brings performance improvements, better RAID 5/6 reliability

#140

Post by evil-olive »

dekhn wrote: post #27
dale_glass wrote: post #19

But is the RAID handling remotely sane yet? See https://arstechnica.com/gadgets/2021/09/examining-btrfs-linu... It has gems such as: * It won't boot on a degraded array by default, requiring manual action to mount it * It won't complain if one of the disks is stale * It won't resilver automatically if a disk is re-added to the array I think the first is the killer. RAID is a High Availability measure. Your system is…

Many folks I know who manage storage don't make the boot volume RAID (redundant)- instead, it's some rapidly duplicatable thing like an NMVE flash containing the root filesystem, and there's a replacement handy. Then you can bring up and bring the full power of userspace to bear on the RAID repair.

why_not_both.jpg

I have 2 NixOS-based NASes that run ZFS. each one has 3 equal-sized 256gb SSDs in addition to the pile o' spinning rust.

each SSD has a small UEFI boot partition, then the rest of the space is cut in half.

the root filesystem is a 3-way ZFS mirror of the first half of the SSDs. the second half of each SSD is another 3-way mirror, this time as the "special" / metadata device for the main hard-drive-backed zpool.

any of the 3 SSDs can fail, and it will boot up and mount the storage perfectly fine. I could also easily upgrade to larger SSDs, in-place and with minimal downtime (zero downtime, if my case had hot-swappable SSD bays)

it would work just as well with only 2 SSDs, but the incremental cost of a 3rd is small enough relative to the whole that I went for it.

Post reply on HN