Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Five Years of Btrfs

markmcb.com

Post reply on HN
Print view View original

101–110 of 240 posts

Re: Five Years of Btrfs

#101

Post by tjoff »

jstrong wrote: post #91
iracic wrote: post #74

Earlier quoted context omitted.

It is possible to put database in state that is "ready" for snapshot, pushing changes to disk and sort of freezing I/O during snapshot.

this is generally not a matter of concern for a copy on write filesystem like zfs, since it's not possible for the file to be in an "in between" state. If a write were in progress, the filesystem would still be pointing to the previous state. Only when the data is written to disk is the pointer moved to the new location.

It very much is a concern. ZFS has no knowledge about the internals of a database, which parts of a file are related to each other etc.

Re: Five Years of Btrfs

#102

Post by toast0 »

vetinari wrote: post #72
Fnoord wrote: post #66

Earlier quoted context omitted.

Hardware RAID can also suffer from this indeed but does ZFS suffer from it as well? With exactly the same impact? AFAIK the filesystem stays consistent on ZFS.

raidz1 is not raid5. From https://pthree.org/2012/12/05/zfs-administration-part-ii-rai... : > ather than the stripe width be statically set at creation, the stripe width is dynamic. Every block transactionally flushed to disk is its own stripe width. Every RAIDZ write is a full stripe write. Further, the parity bit is flushed with the stripe simultaneously, completely eliminating the RAID-5 write hole. So, in the eve…

Raidz1 isn't raid5, but if it mostly solves the same problem for users, without running into the write hole issue, isn't that suggesting we use raidz1 on zfs instead of raid5 on btrfs if we're concerned about unclean shutdowns?

What would we be missing in terms of capabilities by having raidz1 instead of raid5? (Just from the redundancy and performance point of view; let's assume everything else on btrfs and zfs is equal)

Re: Five Years of Btrfs

#103

Post by gravypod »

I've seen a lot of the hacker community focusing on btrfs and zfs but very little focusing on ceph. I think ceph has a lot of the features that we want in a file system and some things that aren't even possible on traditional file systems (per-file redundancy settings) with very little downsides. The setup is a little more complex involving a few daemons to manage disks, balance, monitor, etc. I wish there was something similar to FreeNAS for ceph that only focused on making the experience seemless because I think if it became more popular in the home lab space we'd see lots of cool tools pop up for it.

Re: Five Years of Btrfs

#104

Post by rcthompson »

takeda wrote: post #97
tezzer wrote: post #95

I've had one issue with btrfs that took it off my radar completely. A customer had a runaway issue that filled a btrfs device with unimportant things. We found the errant process and killed it, but apparently if a btrfs device is completely full, you can't delete anything to free up space. File removal requires some amount of free space. Bricked the device, annoyed a customer, back to ext4.

ZFS had this issue (I believe fixed) workaround was to pick up one large file that you wanted to delete and do `echo -n > /the/unimportant/file` once the file was reduced in size to 0, rm started to work again. Not sure if that workaround would work in btrfs, but it worked on ZFS.

What happens if the file has already found its way into a snapshot? Then presumably that command will not free any space.

Re: Five Years of Btrfs

#105

Post by wtfrmyinitials »

geophertz wrote: post #37

Is using btrfs on a personal machine something to do? It seems that all the comments as well as articles about it, just assume you're running it on a server. The ability to add and remove disks on a desktop machine is very tempting.

I've been running it on my desktop for a while and it's been wonderful. I have a cron job set to take a snapshot of the filesystem hourly so if I ever blow a file away or a package upgrade goes wonky I'm back up and running in minutes.

Re: Five Years of Btrfs

#106

Post by proaralyst »

tezzer wrote: post #95

I've had one issue with btrfs that took it off my radar completely. A customer had a runaway issue that filled a btrfs device with unimportant things. We found the errant process and killed it, but apparently if a btrfs device is completely full, you can't delete anything to free up space. File removal requires some amount of free space. Bricked the device, annoyed a customer, back to ext4.

The trick was to insert a USB drive, tell BTRFS it's block storage and delete away. Once you're done you tell it not to use the disk and you're good.

This is why I'm back on ext4 now, too.

Re: Five Years of Btrfs

#107

Post by tezzer »

proaralyst wrote: post #106
tezzer wrote: post #95

I've had one issue with btrfs that took it off my radar completely. A customer had a runaway issue that filled a btrfs device with unimportant things. We found the errant process and killed it, but apparently if a btrfs device is completely full, you can't delete anything to free up space. File removal requires some amount of free space. Bricked the device, annoyed a customer, back to ext4.

The trick was to insert a USB drive, tell BTRFS it's block storage and delete away. Once you're done you tell it not to use the disk and you're good. This is why I'm back on ext4 now, too.

Good trick, I'll save that one. Thanks!

Re: Five Years of Btrfs

#108

Post by louwrentius »

gravypod wrote: post #103

I've seen a lot of the hacker community focusing on btrfs and zfs but very little focusing on ceph. I think ceph has a lot of the features that we want in a file system and some things that aren't even possible on traditional file systems (per-file redundancy settings) with very little downsides. The setup is a little more complex involving a few daemons to manage disks, balance, monitor, etc. I wish there was someth…

I love Ceph, I even wrote an intro about it for those who are not familiar with it.

https://louwrentius.com/understanding-ceph-open-source-scala...

But Ceph is not designed to be a competitor to BTRFS or ZFS. The core vision of Ceph is scalability. If you need petabytes of storage and the performance to scale with it, take a look at Ceph.

I may be totally wrong here, but from what I understand about Ceph, it's not meant as a file system for a single computer. I don't understand the idea of running Ceph on your laptop/desktop. It's possible to run it that way but it defeats it's purpose.

I've build a small lab setup with Ceph:

https://louwrentius.com/my-ceph-test-cluster-based-on-raspbe...

Also, there's the issue of performance, in particular latency. That's a bit of a weak spot of Ceph, from what I can tell. Again, may be wrong. But I found these notes interesting.

https://yourcmc.ru/wiki/Ceph_performance

Re: Five Years of Btrfs

#109

Post by starfallg »

InTheArena wrote: post #31

I went on a quest a few years ago, thinking it would be good for the industry to standardize on a single next generation filesystem for UNIX. I started with ZFS on linux since that seemed to have the most vocal advocates. That lasted about a half year, until a bug in the code resulted in a completely corrupt disk, and I had to restore 4TB of data over a month from offside backups. That plus the licensing confusion ar…

Btrfs is the only FS I used that resulted in complete FS corruption losing nearly all data on disk, not once, but 3 times.

After that, none of the features like compression, snapshots, COW or checksums meant anything to me. I'm much happier with ext4 and xfs on lvm.

Re: Five Years of Btrfs

#110

Post by saalweachter »

xzcat wrote: post #53
pmarreck wrote: post #24

Earlier quoted context omitted.

I have to beg to differ here as I had a different experience that I literally just posted about to Reddit yesterday https://www.reddit.com/r/zfs/comments/eu1qsj/a_tale_of_two_f... tl;dr Unbeknownst to me I had a bad drive cable for an external NVMe enclosure that was causing intermittent I/O errors (only during high drive utilization) that went undetected by BTRFS and slowly corrupted my drive, eventually leading to…

I'll throw in my own anecdote. ZFS on root caused me a significant amount of headache when the proxmox node I was using it on just randomly decided it wasn't going to boot anymore. The ZFS pools were fine, no data was lost, but no amount of messing with it fixed the zfsonroot and it was quite difficult to find quality search results for. And of course it was a weekend where my parents and siblings and in-laws were vi…

I suspect that both BTFS and ZFS are currently good enough under most configurations that most users don't have a problem with whichever they choose, and it's only a tiny fraction that has a really good or bad experience and becomes a rabid advocate based on their anecdotes.
Post reply on HN