Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Five Years of Btrfs

markmcb.com

Post reply on HN
Print view View original

71–80 of 240 posts

Re: Five Years of Btrfs

#71

Post by muxator »

derefr wrote: post #27

A question for HN: what filesystem and/or block-device abstraction layer would you use on a database server, if you wanted to perform scheduled incremental backups using filesystem-level consistent snapshotting and differential snapshot shipping to object storage, instead of using the DBMS’s own replication layer to achieve this effect? (I.e. you want disaster recovery, not high availability.) Or, to put that another…

I do not think it would be a good idea to use file system level snapshotting for backing up a database. The database "knows better" about its internals, and can give more guarantees about the consistency of its data. I would trust a filesystem-levdl backup only as a last resort.

Re: Five Years of Btrfs

#72

Post by vetinari »

Fnoord wrote: post #66
vetinari wrote: post #63

Earlier quoted context omitted.

That one is here to stay, it is a property of software-based RAID. If it bothers you, use UPS.

Hardware RAID can also suffer from this indeed but does ZFS suffer from it as well? With exactly the same impact? AFAIK the filesystem stays consistent on ZFS.

raidz1 is not raid5.

From https://pthree.org/2012/12/05/zfs-administration-part-ii-rai... :

> ather than the stripe width be statically set at creation, the stripe width is dynamic. Every block transactionally flushed to disk is its own stripe width. Every RAIDZ write is a full stripe write. Further, the parity bit is flushed with the stripe simultaneously, completely eliminating the RAID-5 write hole. So, in the event of a power failure, you either have the latest flush of data, or you don't. But, your disks will not be inconsistent.

> There's a catch however. With standardized parity-based RAID, the logic is as simple as "every disk XORs to zero". With dynamic variable stripe width, such as RAIDZ, this doesn't work. Instead, we must pull up the ZFS metadata to determine RAIDZ geometry on every read. If you're paying attention, you'll notice the impossibility of such if the filesystem and the RAID are separate products; your RAID card knows nothing of your filesystem, and vice-versa. This is what makes ZFS win.

Re: Five Years of Btrfs

#73

Post by clSTophEjUdRanu »

InTheArena wrote: post #31

I went on a quest a few years ago, thinking it would be good for the industry to standardize on a single next generation filesystem for UNIX. I started with ZFS on linux since that seemed to have the most vocal advocates. That lasted about a half year, until a bug in the code resulted in a completely corrupt disk, and I had to restore 4TB of data over a month from offside backups. That plus the licensing confusion ar…

I really don't understand the insane hype around ZFS. You can't read any thread that touches on filesystems without the ZFS zealots coming out.

Re: Five Years of Btrfs

#74

Post by iracic »

muxator wrote: post #71
derefr wrote: post #27

A question for HN: what filesystem and/or block-device abstraction layer would you use on a database server, if you wanted to perform scheduled incremental backups using filesystem-level consistent snapshotting and differential snapshot shipping to object storage, instead of using the DBMS’s own replication layer to achieve this effect? (I.e. you want disaster recovery, not high availability.) Or, to put that another…

I do not think it would be a good idea to use file system level snapshotting for backing up a database. The database "knows better" about its internals, and can give more guarantees about the consistency of its data. I would trust a filesystem-levdl backup only as a last resort.

It is possible to put database in state that is "ready" for snapshot, pushing changes to disk and sort of freezing I/O during snapshot.

Re: Five Years of Btrfs

#75

Post by Filligree »

xzcat wrote: post #53
pmarreck wrote: post #24

Earlier quoted context omitted.

I have to beg to differ here as I had a different experience that I literally just posted about to Reddit yesterday https://www.reddit.com/r/zfs/comments/eu1qsj/a_tale_of_two_f... tl;dr Unbeknownst to me I had a bad drive cable for an external NVMe enclosure that was causing intermittent I/O errors (only during high drive utilization) that went undetected by BTRFS and slowly corrupted my drive, eventually leading to…

I'll throw in my own anecdote. ZFS on root caused me a significant amount of headache when the proxmox node I was using it on just randomly decided it wasn't going to boot anymore. The ZFS pools were fine, no data was lost, but no amount of messing with it fixed the zfsonroot and it was quite difficult to find quality search results for. And of course it was a weekend where my parents and siblings and in-laws were vi…

That's really a packaging issue, not a ZFS issue, but I feel your pain.

The best suggestion I can offer is to use a distribution that treats it like a first-class citizen, such as... well, the Ubuntu support is still beta level, so only NixOS for now.

Re: Five Years of Btrfs

#76

Post by asveikau »

clSTophEjUdRanu wrote: post #73
InTheArena wrote: post #31

I went on a quest a few years ago, thinking it would be good for the industry to standardize on a single next generation filesystem for UNIX. I started with ZFS on linux since that seemed to have the most vocal advocates. That lasted about a half year, until a bug in the code resulted in a completely corrupt disk, and I had to restore 4TB of data over a month from offside backups. That plus the licensing confusion ar…

I really don't understand the insane hype around ZFS. You can't read any thread that touches on filesystems without the ZFS zealots coming out.

I don't think I am a zealot, nor a heavy user, but I use it on 1 machine at home (an NFS server running FreeBSD, which I have clients for elsewhere in my house). I came to this idea when I saw some data loss on some magnetic disks in my house, and repairing or even assessing the level of damage was difficult.

My experience is that it's pretty good. The tooling does what it says without a lot of drama. I can scrub while the system is in use and don't notice it mostly. I have seen some small corruptions that it was able to flag for me with specific filenames and fix. Snapshotting and send/receive is also very handy.

I heard some people say they don't like to use it under heavy load. That seems reasonable to me. You're paying costs to get the integrity piece. So it's not for every use or every user. It is very good at what it does, however.

Re: Five Years of Btrfs

#77

Post by KaiserPro »

derefr wrote: post #27

A question for HN: what filesystem and/or block-device abstraction layer would you use on a database server, if you wanted to perform scheduled incremental backups using filesystem-level consistent snapshotting and differential snapshot shipping to object storage, instead of using the DBMS’s own replication layer to achieve this effect? (I.e. you want disaster recovery, not high availability.) Or, to put that another…

AWS, and possibly GCP only allow 1:1 mapping of volumes (publicly, I know AWS allow it under the hood. )

Which makes synchronising snaphots a lot easier (and caching too, but thats another thing entirely.)

They are treated as block storage, so on the outside don't have to worry about what filesystem is running on it. (in practice they have to be a bit aware, so that they don't snapshot unbootable or dirty images, but I assume thats mostly handled by an OS plugin)

TL;DR:

AWS et al snapshots are at the block level. Linux has poorly documented primitives for this.

If you put your VM images on a Filesystem provided by ZFS or BTRFS then you can snapshot your images, without having to buy a SAN, or expensive controller.

ZFS has by far the best documentation. BTRFS's documentation has improved, but the tools are still difficult to use.

Re: Five Years of Btrfs

#78

Post by herf »

abotsis wrote: post #48

It’s worth noting that much of the premise of the article (wanting flexibility) is outdated. Zfs has support for removing top-level raid 0/1 vdevs now. So you can take a raid10 pool, and remove a top level mirror vdev completely. Note that this doesn’t work for raid5/6 vdevs, but as the author points out, those are becoming less and less used because of rebuild time and performance. In addition to the slew of other f…

zfs remove is not a very good implementation - it keeps the old blocks around (as a virtual device) and redirects them to new locations. This is fine for "oops I accidentally added a device" but not great otherwise.

Re: Five Years of Btrfs

#79

Post by yjftsjthsd-h »

clSTophEjUdRanu wrote: post #73
InTheArena wrote: post #31

I went on a quest a few years ago, thinking it would be good for the industry to standardize on a single next generation filesystem for UNIX. I started with ZFS on linux since that seemed to have the most vocal advocates. That lasted about a half year, until a bug in the code resulted in a completely corrupt disk, and I had to restore 4TB of data over a month from offside backups. That plus the licensing confusion ar…

I really don't understand the insane hype around ZFS. You can't read any thread that touches on filesystems without the ZFS zealots coming out.

ZFS is mature/stable, its feature set is basically unmatched (data checksums, compression, atomic snapshots, RAID(0,1,10,5,6), send/receive) by any other option on Linux, and what competition it does have is unstable in some configurations (BTRFS), essentially dead in the water (reiserfs), in early development (bcachefs), or far more complex to manage (gluster, ceph, LVM+XFS). Other than the licensing issue, ZFS is basically a silver bullet.

Re: Five Years of Btrfs

#80

Post by lostlogin »

zionic wrote: post #68
InTheArena wrote: post #31

I went on a quest a few years ago, thinking it would be good for the industry to standardize on a single next generation filesystem for UNIX. I started with ZFS on linux since that seemed to have the most vocal advocates. That lasted about a half year, until a bug in the code resulted in a completely corrupt disk, and I had to restore 4TB of data over a month from offside backups. That plus the licensing confusion ar…

I have a synology NAS on btrfs. One of the best computer purchases I've ever made.

I’ll second this, it’s fantastic. The time it takes to expand when adding a second 16TB is deeply average (8 days) but that’s about it for downsides. It’s the best computer I’ve owned.
Post reply on HN