Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

ZFS, BTRFS, XFS, EXT4 and LVM with KVM – A Storage Performance Comparison (2015)

ilsistemista.net

Post reply on HN
Print view View original

61–70 of 85 posts

Re: ZFS, BTRFS, XFS, EXT4 and LVM with KVM – A Storage Performance Comparison (2015)

#61

Post by ryao »

kinkdr wrote: post #35
matt-attack wrote: post #5

What is with BTRFS? It stands out in every single test (and not in a good way).

Maybe it is not there yet, but given the momemtum it has, my gut feeling is that it will catch up eventually and be our default fs one day.

The reality is that many are not interested in waiting and adopt ZFS permanently. This was my experience ~6 years ago. Had btrfs worked better then, I would probably have been a btrfs contributor today.

Of what I know now about the internals of each, I am glad that I went with ZFS. The decisions in ZFS are more robust in terms of reliability and performance. e.g. The merkle tree, the use of 256-but checksums, mandatory ditto blocks, the ARC algorithm, the intent log, default ditto blocks, an variable height, indirect block tree, etcetera.

Re: ZFS, BTRFS, XFS, EXT4 and LVM with KVM – A Storage Performance Comparison (2015)

#62

Post by Zardoz84 »

Interesting. But in our case, we are running Hyper-V VMs (son sadly NTFS on host), were the VMs uses EXT4, and a few uses BTRFS.

The only problem that we had, was with a uncontrolled poweroff a year ago. Looks that BRTFS had some trouble recovering from it, but we manage to restore all data from the partitions. However, the flexibility that offer BTRFS (transparent compression, increasing hard disk space on demand, etc) is really nice. I hope that it improve more, so we can use it without any issues.

Re: ZFS, BTRFS, XFS, EXT4 and LVM with KVM – A Storage Performance Comparison (2015)

#63

Post by MertsA »

ryao wrote: post #46
josefbacik wrote: post #21

Earlier quoted context omitted.

NOCOW is horribly expensive because we still have to go check and make sure that there are no snapshots pointing at the changing extents. It only solves the fragmentation issue, and if you don't prealloc your image it doesn't even do that.

Are you sure that disabling CoW solves btrfs' fragmentation issue when you pre-allocate the image? If the volume is snapshotted regularly, CoW should be active once on each extent once until the next snapshot reactivates it. That should mean that the fragmentation would still occur, although more slowly. Is that correct or am I misunderstanding something? I am only superficially familiar with btrfs internals, but I d…

Disabling CoW disables snapshotting.

Re: ZFS, BTRFS, XFS, EXT4 and LVM with KVM – A Storage Performance Comparison (2015)

#64

Post by MertsA »

ars wrote: post #4

This is more than a year old. I've been told that btrfs has improved greatly in that time.

This is more than a year old and the hardware this was tested on wasn't even that exceptional when it was released 7 years ago. I'm kind of assuming that the tester did something strange for BTRFS in particular because his results disagree with every other benchmark out there. Using BTRFS on top of software RAID 10 is also inappropriate as this should be done by creating a filesystem containing all four devices.

That being said I'd love to see more benchmarks of BTRFS compared to other filesystems and on hardware that isn't so archaic. I think it's safe to say that this article is not representative of reality as Phoronix has tons of benchmarks that all don't have nearly this big of difference between BTRFS and Ext4. Here's an article that's even more outdated than the parent and it still shows BTRFS performing acceptably across the board.

https://www.phoronix.com/scan.php?page=article&item=linux_ra...

Re: ZFS, BTRFS, XFS, EXT4 and LVM with KVM – A Storage Performance Comparison (2015)

#65

Post by josefbacik »

rurban wrote: post #60
josefbacik wrote: post #23

Earlier quoted context omitted.

ZFS has a 7 year head start on BTRFS and has traditionally had about 5 times as many core contributors at any given time so I imagine they've solved this in some novel way by now.

5x more core ZFS contributors sounds to me worse and not better than BTRFS. The more the worse usually.

Not sure how that tracks. These file systems are huge code bases with lots of problems and areas for improvement. When you only have 2-4 people who understand most of it at a given time things move slowly. If I could tell somebody to go fix balance my life would be a whole lot easier, but in reality we end up being interrupt driven and have to prioritize differently.

Re: ZFS, BTRFS, XFS, EXT4 and LVM with KVM – A Storage Performance Comparison (2015)

#66

Post by mrmondo »

- Very old kernel used (3.10!), makes me wonder how old the packages like btrfs-progs are as well.

- BTRFS not mounted with compression (compress=lzo)

- Don't use QCOW2, just don't, it's slow and you're just adding extra layers where you don't need to.

It would be interesting to see you re-run these tests using a modern kernel, say at least 4.4 and either raw block devices or logical volumes along with mounting BTRFS properly with the compress=lzo option

Re: ZFS, BTRFS, XFS, EXT4 and LVM with KVM – A Storage Performance Comparison (2015)

#67

Post by tremon »

Zardoz84 wrote: post #62

Interesting. But in our case, we are running Hyper-V VMs (son sadly NTFS on host), were the VMs uses EXT4, and a few uses BTRFS. The only problem that we had, was with a uncontrolled poweroff a year ago. Looks that BRTFS had some trouble recovering from it, but we manage to restore all data from the partitions. However, the flexibility that offer BTRFS (transparent compression, increasing hard disk space on demand, e…

increasing hard disk space on demand

Note that ext4 on LVM can do this as well, you just need to plan for it in advance. All of my systems are configured as fs-on-lvm, even the single-disk ones. I've found it's just less hassle: I can do all storage migration or storage expansion without ever taking the machine offline.

Re: ZFS, BTRFS, XFS, EXT4 and LVM with KVM – A Storage Performance Comparison (2015)

#68

Post by rodgerd »

biokoda wrote: post #9
josefbacik wrote: post #7

Earlier quoted context omitted.

Kvm (or any overwrite workload for that matter) is the worst possible workload for btrfs because of COW. We have ideas to address this but honestly it's not high on the list.

What kinds of workloads is it designed for then?

Honestly, after more than two years of running btrfs on data I care about, my answer would be "testing your backups".

Re: ZFS, BTRFS, XFS, EXT4 and LVM with KVM – A Storage Performance Comparison (2015)

#69

Post by zbraniecki »

josefbacik wrote: post #7
matt-attack wrote: post #5

What is with BTRFS? It stands out in every single test (and not in a good way).

Kvm (or any overwrite workload for that matter) is the worst possible workload for btrfs because of COW. We have ideas to address this but honestly it's not high on the list.

Is brtfs a good choice from the power consumption standpoint? I'm on a laptop and trying to maximize my battery life, and am wondering if my choice of brtfs limits it.

Re: ZFS, BTRFS, XFS, EXT4 and LVM with KVM – A Storage Performance Comparison (2015)

#70

Post by versteegen »

MertsA wrote: post #63
ryao wrote: post #46

Earlier quoted context omitted.

Are you sure that disabling CoW solves btrfs' fragmentation issue when you pre-allocate the image? If the volume is snapshotted regularly, CoW should be active once on each extent once until the next snapshot reactivates it. That should mean that the fragmentation would still occur, although more slowly. Is that correct or am I misunderstanding something? I am only superficially familiar with btrfs internals, but I d…

Disabling CoW disables snapshotting.

(I'm not an expert, just a user) Snapshotting effectively disables NOCOW. Snapshotting always works, and needs COW to do so. Which means ryao is right that fragmentation will occur. btrfs doesn't copy a NOCOW file in its entirety if it's in a snapshot, or I would have run out of disk space by now. (I really need to move my VM images to a non-backed-up subvolume.)
Post reply on HN