Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Five Years of Btrfs

markmcb.com

Post reply on HN
Print view View original

41–50 of 240 posts

Re: Five Years of Btrfs

#41

Post by unixhero »

derefr wrote: post #27

A question for HN: what filesystem and/or block-device abstraction layer would you use on a database server, if you wanted to perform scheduled incremental backups using filesystem-level consistent snapshotting and differential snapshot shipping to object storage, instead of using the DBMS’s own replication layer to achieve this effect? (I.e. you want disaster recovery, not high availability.) Or, to put that another…

Zfs

Btrfs

See various "Private Cloud" Linux distributions for implementation examples of this. Such as Proxmox which does it out of the box on ZFS and soon on btrfs too.

Re: Five Years of Btrfs

#43

Post by jxcl »

pantalaimon wrote: post #39
pojntfx wrote: post #3

Love using Btrfs; the is no better filesystem than it nowadays that it's reliability issues have been fixed.

> nowadays that it's reliability issues have been fixed Is this also true for RAID5/6?

This issue has its own wiki page on the BTRFS wiki:

https://btrfs.wiki.kernel.org/index.php/RAID56

So, no, that particular issue hasn't been fixed.

Re: Five Years of Btrfs

#44

Post by sp332 »

pantalaimon wrote: post #39
pojntfx wrote: post #3

Love using Btrfs; the is no better filesystem than it nowadays that it's reliability issues have been fixed.

> nowadays that it's reliability issues have been fixed Is this also true for RAID5/6?

There are some big caveats there. https://btrfs.wiki.kernel.org/index.php/RAID56

Re: Five Years of Btrfs

#45

Post by Fnoord »

jbotz wrote: post #20
ailideex wrote: post #5

Earlier quoted context omitted.

What about the reliability? Are many people losing data with Btrfs?

As best I can tell, reports of data loss on btrfs are all from the early 20-teens; after about 2014 or so I can't find anyone who claims to have lost data due to a btrfs bug on an up-to-date system.

RAID5 on btrfs has a write hole last time I checked. Bug has been around forever, and was around in 2014 for sure.

Phoronix has some thorough performance comparisons between Ext4fs, Btrfs, XFS, and ZFS.

Re: Five Years of Btrfs

#46

Post by izacus »

It's also worth noting that Synology uses btrfs as an option to do checksumming and snapshots on their NAS devices.

They're still using their own RAID layer though.

Re: Five Years of Btrfs

#47

Post by polskibus »

StreamBright wrote: post #40
derefr wrote: post #27

A question for HN: what filesystem and/or block-device abstraction layer would you use on a database server, if you wanted to perform scheduled incremental backups using filesystem-level consistent snapshotting and differential snapshot shipping to object storage, instead of using the DBMS’s own replication layer to achieve this effect? (I.e. you want disaster recovery, not high availability.) Or, to put that another…

>> Or, to put that another way: what are AWS and GCP using in their SANs (EBS; GCE PD) that allows them to take on-demand incremental snapshots of SAN volumes, and then ship those snapshots away from the origin node into safer out-of-cluster replicated storage (e.g. object storage)? As far as I know AWS does not use SANs because they consider it as anti-pattern. Most backups land on S3 because of reliability and pric…

So how is S3 implemented? Does it reuse any publicly available open source component?

Re: Five Years of Btrfs

#48

Post by abotsis »

It’s worth noting that much of the premise of the article (wanting flexibility) is outdated. Zfs has support for removing top-level raid 0/1 vdevs now. So you can take a raid10 pool, and remove a top level mirror vdev completely. Note that this doesn’t work for raid5/6 vdevs, but as the author points out, those are becoming less and less used because of rebuild time and performance.

In addition to the slew of other features Btrfs is missing (send/recv, dedup, etc) zfs allows you to dedicate something like an Intel optane (or other similar high write endurance, low latency ssd) to act as stable storage for sync writes, and a different device (typically mlc or tlc flash) to extend the read cache.

Re: Five Years of Btrfs

#49

Post by 3fe9a03ccd14ca5 »

cyphar wrote: post #32
alyandon wrote: post #17

I use btrfs in raid1 mode and the ability to shrink/grow/add/remove devices at will without data loss or extended downtime led me to choose btrfs over zfs on my home servers.

You can grow and add/remove raid1 devices (mirror vdevs) in ZFS without any significant work or downtime. Shrinking does require a bit more work, but depending on your setup it can be done fairly painlessly with send/recv (and shrinking is usually not something which is a very common administrative operation).

How? My understanding is that you create a new vdev and add the old vdev as a device, basically recursively creating volumes with each new device you add.

Re: Five Years of Btrfs

#50

Post by xzcat »

cyphar wrote: post #32
alyandon wrote: post #17

I use btrfs in raid1 mode and the ability to shrink/grow/add/remove devices at will without data loss or extended downtime led me to choose btrfs over zfs on my home servers.

You can grow and add/remove raid1 devices (mirror vdevs) in ZFS without any significant work or downtime. Shrinking does require a bit more work, but depending on your setup it can be done fairly painlessly with send/recv (and shrinking is usually not something which is a very common administrative operation).

"fairly painlessly" and "without significant work or downtime" doesn't sound like it lines up with btrfs's, which I would describe as "one command and zero downtime (just some io load if you rebalance immediately)" for both operations. btrfs is also mainline, which increases how painless it is to use.

BTRFS does have some scary stories from earlier in its development, and true raid5 seems like it's unlikely to be safe for quite a while, but raid1 and "normal" fs usage has been rock solid in my experience. The only time I've ever had an issue was probably 4 years ago at this point, and it was solved by just booting an Arch live iso and running a btrfs command that was basically "fix exactly the bug that your error message indicates". I don't remember exactly what it is, something about two sizes not matching, but googling the text it showed at boot led me directly to the command to fix it. Certainly dramatically less trouble than I've ever had when hardware RAID goes south.

I do agree that modern lvm does probably compete with btrfs, but again you're trading how dang simple btrfs raid1 is to manage for monkeying with partitions in lvm in exchange for ~some? performance.

IMO ZFS is in a weird spot where I don't know where I'd use it. It's too complicated/annoying to admin for me to want to run it in my basement for myself/my family, and for anything bigger or more professional I'd use ceph or a problem-domain-specific storage system (HDFS, clickhouse, aws, etc).

Post reply on HN