Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Linux 5.10 BTRFS performance regression

reddit.com

Post reply on HN
Print view View original

151–160 of 202 posts

Re: Linux 5.10 BTRFS performance regression

#151

Post by enz »

hda111 wrote: post #135
sethhochberg wrote: post #94

Earlier quoted context omitted.

Depending on how you store this data (the number of physical disks involved) you may also want to explore ZFS. ZFS and BTRFS have pretty comparable features, but given how battle-hardened ZFS is, and its ever-increasing integration into the Linux ecosystem, I'm seeing less and less reason to prefer BTRFS. Both do checksumming and regular scrubbing to help detect bitrot, but both need sufficient redundancy configured…

There is exactly one distro that comes with ZFS, so it's a bit exaggeration to say it's well integrated with Linux, considering how much effort is applied by kernel developers to make it even harder to use ZFS. And this one distro that includes ZFS is just one lawsuit away from removing it again.

I am testing OpenZFS with Ubuntu 20.04 LTS right now. OpenZFS seems pretty solid so far (I tried various scenarios of failure/corruption) and Ubuntu seems to actually care about ZFS support and its integration in Linux.

Re: Linux 5.10 BTRFS performance regression

#152

Post by mike256 »

gregkh wrote: post #89
tuldia wrote: post #81

Earlier quoted context omitted.

Every software has bugs, and is easier to criticize than to help. Don't focus on the negativity of HN and keep it up! Thanks for your hard work, Greg!

If people don't report bugs, we don't know they are there as it "works for me!". This isn't "negativity", this is people not understanding how the process works :) And you're welcome!

As every patch nowadays means that yet another symbol gets his gpl-only tag, no I don't report bugs...

Re: Linux 5.10 BTRFS performance regression

#153

Post by yokem55 »

yjftsjthsd-h wrote: post #83
NavinF wrote: post #72

Earlier quoted context omitted.

ZFS is not harder to set up imo. It's as easy as `sudo apt install zfsutils-linux` and `zpool create poolname raidz2 disk1 disk2 disk3` I've been bitten by btrfs back when it was the new hot FS, so I'll just stick to ZFS until a distributed (multiple machines) equivalent of ZFS comes along.

> until a distributed (multiple machines) equivalent of ZFS comes along. Ceph?

That's exactly what they want. It's got all the checksummed replication features that makes zfs awesome and it's pretty rock solid. It's even pretty resilient to dumb mistakes from the admin (ahem). Unfortunately ceph is primarily developed for folks running massive petabyte scale clusters, so setting up a home lab micro-cluster is still a fair bit more difficult than it needs to be, even with the new cephadm stuff. To wit, figuring out what the best practices for setting things up to run a small scale cluster is often guesswork and the pitfalls aren't always obvious.

Re: Linux 5.10 BTRFS performance regression

#154

Post by mike256 »

viraptor wrote: post #127
washadjeffmad wrote: post #28

Earlier quoted context omitted.

ZFS's killer feature is religious inspiration. Write holes during resilvers are healthy and natural, but the same in a software RAID 5/6 configuration on BTRFS is unacceptably risky. Brought to you by the same people who don't think SMR disks are used with ZFS in enterprise.

Raid5 on btrfs is not and has not been production ready. It's available in the code, but with appropriate warnings. You're not supposed to be using it, rather than it being "unacceptably risky".

That's the problem with btrfs. Raid5/6 still isn't production ready and loses data with cheap consumer ssd which could to disappear/reappear in a running system. On the other side raidz/z2 works rock solid. Of course I could use btrfs with mirrors instead but that's the reason why I use cheap consumer ssds. I can't spend that much money just to use btrfs so I stay with zfs.

Re: Linux 5.10 BTRFS performance regression

#155

Post by swinglock »

throwii wrote: post #92
ldrndll wrote: post #61

Earlier quoted context omitted.

I think the issue is that you haven’t understood the issue, and just proposed using Rust. Don’t get me wrong, I’m a huge fan of Rust, but seeing people blindly suggest it as a panacea gets frustrating for people. The reality is that when people talk about Rust being fact, it’s because a decently optimised Rust program might be comparable to an equivalent C program. C is generally not the problem when it comes to perf…

> C is generally not the problem when it comes to performance issues in the kernel, which means rewriting that thing in Rust wouldn’t magically make it faster. Agreed. > The point being, it’s not clear yet, and to solve that requires understanding the problem, not effecting a needless rewrite in a new language. Which is why the first question is why btrfs has issues others do not have. Some mention its CoW architectu…

Btrfs is really only comparable to ZFS, which is also CoW, feature rich and written in C, but has been considered stable for a great many years.

Re: Linux 5.10 BTRFS performance regression

#156

Post by ksk »

teekert wrote: post #145
ksk wrote: post #136

Earlier quoted context omitted.

What software do you use ? We use bwa/samtools/varscan/bcftools and we've reprocessed old data right from the fastq files a few times. So far we haven't gotten different results.

I encounter an error that I can't currently find exactly but it happens when running mismatch_profile from the RSeQC package. It reports a mismatch in the length of the cigar notation and the actual read inside a BAM file. Lately I have become 99% sure that this sometimes happens in files that did not show this error before (we reprocess a lot of data from time to time). I suspect it is bit flips because I think bit…

Hmm, yeah that is odd! Maybe its a bug in the alignment tool?

Re: Linux 5.10 BTRFS performance regression

#157

Post by minot »

simion314 wrote: post #130
therealrootuser wrote: post #119

Earlier quoted context omitted.

Over the last 15 years, I've run a lot of distros on a lot of different hardware. On desktop, Fedora has been by far the most reliable. Ubuntu LTS, as far as I can tell from using it for work, just means that annoying bugs that have been fixed in the upstream never get backported. Case in point: Bluetooth on Ubuntu LTS has always been much less reliable for me than on Fedora. Disclaimer: This is only my anecdotal exp…

You can't have it both ways, you can't get only the bug fixes without the new bugs , except on LTS you can if you know how experiment with getting a new kernel or video driver but not updating your DE,

> getting a new kernel

> you can't get only the bug fixes without the new bugs

> if you know how experiment with getting a new kernel or video driver

I would agree with you if I was on my old machine with an nVidia graphics card which for what it is worth my new machine is a much better experience with AMD processor and integrated graphics but still how do you explain audio in failing to work as soon as I upgraded to Fedora 33 / kernel 5.8 and it (mostly) resolving itself when upgrading to 5.9 kernel? Is this a kernel bug or not? What caused it? What can we do to prevent it from happening in the future?

Re: Linux 5.10 BTRFS performance regression

#158

Post by enz »

bluedino wrote: post #50
m1gu3l wrote: post #5

The ratio of bad to good posts about BTRFS always made me wonder why anyone would roll with it. Don’t put your fs in crazy.

Synology has been using BTRFS for a while, I haven't had any issues with it. https://www.synology.com/en-global/dsm/Btrfs

It's interesting to compare with QNAP, a competitor of Synology, that does not use Btrfs (they stick to ext4 with mdadm) and they explained why: https://www.qnap.com/solution/qnap-ext4/en/

However, I am really not sure if the facts they provide are still relevant today. Like always with commercial documents, that must be taken with a grain of salt.

Re: Linux 5.10 BTRFS performance regression

#159

Post by enz »

thomond wrote: post #22
m1gu3l wrote: post #5

The ratio of bad to good posts about BTRFS always made me wonder why anyone would roll with it. Don’t put your fs in crazy.

Didn't Fedora transition to BTRFS as default in 33? I'm eager to see reports from that.

It did, but as far as I know it's effective on fresh installs only. It would be too risky to migrate automatically to Btrfs from ext4 during the 32->33 upgrade. I suspect a lot of users upgraded to 33 from 32 without seeing any problem with sticking with ext4 (including me). As a result, a lot of users are still using ext4.

Re: Linux 5.10 BTRFS performance regression

#160

Post by GayChromulent »

gregkh wrote: post #56
esgwpl wrote: post #14

This version has got to be the worst kernel released in a while in terms of regression, from AMDGPU null pointer dereference crash[0] to f2fs data corruption bug[1] and now this. Fixes for these are on their way as far as I can tell but since the stable team are probably on Christmas vacation it might take a while. [0] https://bbs.archlinux.org/viewtopic.php?pid=1943906#p1943906 [1] https://bugzilla.kernel.org/show_b…

I get a vacation? Hah!

Hahaha, found the faggots. Everyone knows that anyone who cheers greg croan heartman is a faggot, exept his well established wife. What linus and he does in the sauna is not up to us, but, I mean everyone who comments under here is a full blooded faggot. Which is not wrong.
Post reply on HN