Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Five Years of Btrfs

markmcb.com

Post reply on HN
Print view View original

171–180 of 240 posts

Re: Five Years of Btrfs

#171

Post by tezzer »

Zardoz84 wrote: post #169
tezzer wrote: post #95

I've had one issue with btrfs that took it off my radar completely. A customer had a runaway issue that filled a btrfs device with unimportant things. We found the errant process and killed it, but apparently if a btrfs device is completely full, you can't delete anything to free up space. File removal requires some amount of free space. Bricked the device, annoyed a customer, back to ext4.

this wasn't fixed years ago ? I had a BTRFs partition filled by a rogue process and it not bricked. it allow me to remove the junk files without any issue. And I'm talking of a Ubuntu 14.04 Ltd server

This was 14.04 as well, on a Tegra (armhf) system.

eta: looked up the ticket, customer reported "when trying to delete any files, even as root, btrfs says "cannot remove"", field engineering observed the same.

Re: Five Years of Btrfs

#172

Post by tw04 »

InTheArena wrote: post #31

I went on a quest a few years ago, thinking it would be good for the industry to standardize on a single next generation filesystem for UNIX. I started with ZFS on linux since that seemed to have the most vocal advocates. That lasted about a half year, until a bug in the code resulted in a completely corrupt disk, and I had to restore 4TB of data over a month from offside backups. That plus the licensing confusion ar…

Legitimately curious what the ZFS bug was. I’ve not heard of a TFDL bug in zfs for a Loooong time.

The reason Synology btrfs is mostly solid is because they refused to ever use the btrfs raid layer. But the second you move to btrfs on LVM you lose a large portion of the supposed benefits.

Having used both, never lost data on zfs and I’ve been using it since it was released and have had it save me from silent data corruption. BTRFS hasn’t ever lost me an entire file system, but I’ve definitely lost files.

Re: Five Years of Btrfs

#173

Post by machawinka »

SEJeff wrote: post #137
thfuran wrote: post #120

Earlier quoted context omitted.

I think the probable cause is that it's not common bugs that cause the corruption but uncommon ones. Most of the time, they work fine. But you really want a stronger guarantee than that out of your filesystem.

Historically, the biggest bugs in btrfs were when you came close to filling up the filesystem. For the longest time, you'd get -ENOSPC (no space left) even when you had many Gb of space left due to really bad metadata and block level space usage.

This is my experinence too. Works great with lots of free space, as soon as space gets tight, performance deteriorates really fast. Nevertheless, for me it has been worthy.

Re: Five Years of Btrfs

#174

Post by Aldipower »

derefr wrote: post #27

A question for HN: what filesystem and/or block-device abstraction layer would you use on a database server, if you wanted to perform scheduled incremental backups using filesystem-level consistent snapshotting and differential snapshot shipping to object storage, instead of using the DBMS’s own replication layer to achieve this effect? (I.e. you want disaster recovery, not high availability.) Or, to put that another…

Assuming you can afford 2 machines this setup works pretty well for me.

Primary DB-Server -> XFS on LVM with a LVM caching SSD

Secondary (write-only) mirror DB-Server -> ZFS

The DB is replicating automatically to the secondary server by the database internal replication features. At the secondary I am then able to lock the DB temporarily, doing a ZFS snapshot and maybe could do a ZFS send/receive afterwards without affecting the primary server.

ZFS is great for doing snapshots and archiving of huge amount of data, but it's very very bad for production databases in terms of performance. Most database aren't designed to deal with the CoW feature of ZFS, which leads to a very bad write performance and database fragmentation in the end.

Re: Five Years of Btrfs

#175

Post by danudey »

packetlost wrote: post #160
RX14 wrote: post #116

Earlier quoted context omitted.

It seems a lot of people have these stories, and then people like me and OP who have had btrfs survive the most fucked up situations (I've had a btrfs nas built on "random drives I've had lying around" and abused it for 5 years and had 0 bugs at all). I'm not sure what causes it, but there seems to be an effect where btrfs loves you or hates you and few people with mixed experiences regarding data loss. One possible…

In my experience, btrfs is very fragile in power loss or kernel crash/panic scenarios. It very consistently causes soft lockups on file read/writes after power loss until you run a `brtfs check --repair` on it. My experience is mostly on Arch, so it's not a case where it's out of date and missing patches.

This was my experience. We had a brief power outage at work and my btrfs (root) partition was toast. Spent a whole day rebuilding my system afterwards. Will definitely not go that route again.

The only difference is that none of the repair tools were able to recover the filesystem, but I was able to dump the files themselves to a new disk to recover them. Really not sure why, it was very strange.

Re: Five Years of Btrfs

#176

Post by danudey »

SEJeff wrote: post #137
thfuran wrote: post #120

Earlier quoted context omitted.

I think the probable cause is that it's not common bugs that cause the corruption but uncommon ones. Most of the time, they work fine. But you really want a stronger guarantee than that out of your filesystem.

Historically, the biggest bugs in btrfs were when you came close to filling up the filesystem. For the longest time, you'd get -ENOSPC (no space left) even when you had many Gb of space left due to really bad metadata and block level space usage.

I'm a huge Mac fanboy, but APFS really kicks me in the teeth sometimes. Aside from things like snapshots, clones, etc. not being accessible to users (well, not really), or being able to create subvolumes at specific mount points which forget those mount points next reboot, it had an extremely strange behavior (possibly relating to snapshots/CoW?) where once it was full, it stayed full forever until you rebooted.

Basically, any time a runaway process filled my disk, I just had to hard-reboot and hope I didn't have any unsaved work or state that I needed to preserve.

Really makes me hope that Apple is going to further extend APFS to not just be baby's first CoW volume-management filesystem.

Re: Five Years of Btrfs

#177

Post by danudey »

somehnguy wrote: post #141
bbatha wrote: post #85

Earlier quoted context omitted.

Also don't underrate good documentation and easy to use tooling.

This was the killer feature for me. I had been wanting to try ZFS on my home NAS for a while (for snapshotting/redundancy/data integrity) and finally got enough disks that it made sense. I wasn't looking forward to learning what I presumed to be a very complicated system though. About 15 minutes into my research for setting up and maintaining a ZFS filesystem and I just went - wait thats it? So incredibly simple and…

ZFS is incredibly easy to learn to use, whereas btrfs is quite complicated to learn/use, and even more so if you've used ZFS since a lot of things are either just different enough to be weird, or so different that it makes no sense.

Examples: ZFS snapshots can be recursive (-r) or not, whereas on btrfs they cannot be recursive; in discussions I've seen, this is mentioned as "a feature", since you can create a subvolume for data that you don't want to be part of the snapshot, but it also prevents you from dividing up a logical heirarchy into multiple behaviours (compression vs. not, block size, etc.).

Re: Five Years of Btrfs

#178

Post by paulddraper »

RX14 wrote: post #116
starfallg wrote: post #109

Earlier quoted context omitted.

Btrfs is the only FS I used that resulted in complete FS corruption losing nearly all data on disk, not once, but 3 times. After that, none of the features like compression, snapshots, COW or checksums meant anything to me. I'm much happier with ext4 and xfs on lvm.

It seems a lot of people have these stories, and then people like me and OP who have had btrfs survive the most fucked up situations (I've had a btrfs nas built on "random drives I've had lying around" and abused it for 5 years and had 0 bugs at all). I'm not sure what causes it, but there seems to be an effect where btrfs loves you or hates you and few people with mixed experiences regarding data loss. One possible…

> an effect where btrfs loves you or hates you

Same thing happens with operating systems.

Re: Five Years of Btrfs

#179

Post by paulddraper »

twic wrote: post #161
clSTophEjUdRanu wrote: post #73

Earlier quoted context omitted.

I really don't understand the insane hype around ZFS. You can't read any thread that touches on filesystems without the ZFS zealots coming out.

Eh, i'm waiting for them to rewrite it in Rust.

Poe's law

Re: Five Years of Btrfs

#180

Post by paulddraper »

wmf wrote: post #168
tjoff wrote: post #127

Earlier quoted context omitted.

As do filesystems. Yet I've seen anyone argue that cutting the power is the recommended way of doing backups. In fact the opposite, make sure to use an UPS just so that you can shutdown cleanly in the unfortunate event. For example: https://blogs.oracle.com/paulie/backing-up-mysql-using-zfs-s...

Some people came up with the idea of "crash-only software", arguing that it's better to maintain one code path (recovering from a crash) than two (clean start and recovery), but it hasn't caught on that much. https://www.usenix.org/legacy/events/hotos03/tech/full_paper...

Google follows this, I believe
Post reply on HN