Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Bcachefs: “the COW filesystem for Linux that won't eat your data”

bcachefs.org

Post reply on HN
Print view View original

71–80 of 110 posts

Re: Bcachefs: “the COW filesystem for Linux that won't eat your data”

#71

Post by __david__ »

teraflop wrote: post #51
Ajedi32 wrote: post #19

Earlier quoted context omitted.

IIRC, doesn't BTRFS also allow you to do cool things like change RAID levels dynamically? (E.g. You can be running a 2-disk RAID-1 array, pop in another disk and tell BTRFS to make it a RAID-5 array instead, then a year later pop in 2 more disks and switch to RAID6, all with no downtime.) I imagine that wouldn't be possible if you were doing RAID at a different layer.

Linux's built-in software RAID (implemented at the block level) has supported online RAID level changes for many years. Check out the "grow mode" section of the mdadm(8) man page sometime; it goes into great detail about which operations are supported.

Yeah, shout out to mdadm and the md driver. I've been using it for years and years and it's rock solid. Being able to grow arrays (online) and convert plain drives into mirror sets is great. I feel that md and lvm are underappreciated by a large number of Linux users…

Re: Bcachefs: “the COW filesystem for Linux that won't eat your data”

#72

Post by __david__ »

ysleepy wrote: post #5

While I really like these sort of file systems, I'm not holding my breath. This isn't a simple filesystem project, but plays in the next-gen space ZFS opened up. There will be a lot to do, especially IO scheduling, RAID safety with shitty drive firmwares, consistency guarantees with fsync/partial flushes etc. I'm pessimistic about it being mainlined in the near future, the core team will be weary of a second btrfs. W…

I'm more optimistic. What you may or may not know is that bcachefs is a tweak on bcache, which has already been mainlined and is pretty stable (I've personally been running bcache for a couple years on my home linux machine).

The point is a lot of the things you bring up are already covered by bcache. Bcachefs "just" adds a filesystem layer on the bcache tree structure.

Re: Bcachefs: “the COW filesystem for Linux that won't eat your data”

#73

Post by sargun »

I'm looking forward to BcacheFS. ZFS on Linux is great when it works well, but it's an absolute pain when it breaks. Not only does it taint the kernel, but it doesn't really mesh very well in the kernel due to the usage of the SPL -- a layer used to convert Linux APIs to Solaris Kernel APIs. In addition, ZFS doesn't use as much native Linux memory management as I'd like, instead it manages its own pool of memory. This makes troubleshooting more difficult. This mechanism is further aggravated with the use of the kmem cgroup.

For example, if you have a dirty page in a cgroup, and the cgroup OOMs, the kernel will trigger writes. If any of these writes require memory allocations, they'll probably fail since the current cgroup is OOM. ZFS subsequently gets stuck in an infinite loop, and locks up. See: https://github.com/zfsonlinux/zfs/issues/5535

I understand that a lot of ZFS works comes from LLNL & government funding. I'm not blaming them, as it works for their use case of machines that are running dedicated, controlled workloads.

We're experimenting with Btrfs, and we'll see how it goes.

Re: Bcachefs: “the COW filesystem for Linux that won't eat your data”

#74

Post by gigatexal »

tylerjd wrote: post #60
gigatexal wrote: post #56

Earlier quoted context omitted.

ZFS gets you all of that and raid and snapshots and has been tested time and time again. Why run BRTFS then?

ZFS is not GPL? I dunno, I like feeling safe when I do kernel upgrades knowing even if for whatever reason my ZFS module doesn't compile/work under the new kernel, I won't be left without a root FS. I have been running BTRFS for 5 years with very little issues, and enjoy not having to compile a new dkms module with every kernel.

Ahhh yes that is the one ma in n point I missed. Can't argue with that. I've just never had issues upgrading as the zfs on Linux project us fantastic.

Re: Bcachefs: “the COW filesystem for Linux that won't eat your data”

#75

Post by phs318u »

This takes me back. 9 years ago I was playing around with ZFS COW and OS X sparse bundle containers to host disks images for multiple "versions" (exploiting CoW) of the same VM image. I wrote up an article on what I was doing [1]. Never persevered though as it was a bit too fragile (at that time ZFS on OS X was not at all ready for prime time).

Funny, but every so often I wonder what it might be like in a parallel world where Apple bought Sun instead of Oracle.

[1] http://macoverdrive.blogspot.com.au/2008/10/using-zfs-to-man...

Re: Bcachefs: “the COW filesystem for Linux that won't eat your data”

#76

Post by Dylan16807 »

gigatexal wrote: post #56
rrauenza wrote: post #31

Earlier quoted context omitted.

I've been using btrfs since about 2014 on CentOS7. I only use it with mirroring+compression. No snapshots. I mostly went with it for data scrubbing and compression. My experience has been mixed, but haven't had any data loss. There was a bug for a while regarding free space so occasionally the system would seem to be full but wasn't ....and it was a real pain to correct it. I now have a cron job that does a monthly b…

ZFS gets you all of that and raid and snapshots and has been tested time and time again. Why run BRTFS then?

BTRFS lets you make CoW copies of files. You can even retroactively merge the blocks that store identical files. BTRFS also makes it not a giant pain to remove a file from snapshots.

ZFS does seem to work better overall, but I wouldn't call either filesystem great at this point in time.

Re: Bcachefs: “the COW filesystem for Linux that won't eat your data”

#77

Post by Zancarius »

tylerjd wrote: post #60
gigatexal wrote: post #56

Earlier quoted context omitted.

ZFS gets you all of that and raid and snapshots and has been tested time and time again. Why run BRTFS then?

ZFS is not GPL? I dunno, I like feeling safe when I do kernel upgrades knowing even if for whatever reason my ZFS module doesn't compile/work under the new kernel, I won't be left without a root FS. I have been running BTRFS for 5 years with very little issues, and enjoy not having to compile a new dkms module with every kernel.

ZFS with DKMS is a disaster, at least in my experience. Honestly, I can't recommend ZoL unless you're running a distro with relatively stable kernel releases that don't change substantially or that happens to be supported by ZoL with binary packages. ZoL on Arch was... trying at times. It worked great, but my paranoia meant that I ended up adding the kernel to IgnorePkg to force manual kernel updates (mostly for my own memory). But then, it also meant having to build all of the ZFS packages (including SPL) tied to that specific version. This usually meant waiting until the AUR packages were updated as I figured that indicated someone else must have tested ZFS on that specific kernel version.

I remembered thinking DKMS might solve the problem, but I ended up having to use recovery media just to get an environment to reinstall an older kernel and let DKMS do its thing after a botched update started provoking panics. I suspect a version mismatch based on the errors but never investigated it beyond fixing the problem and moving to the prebuilt modules. Things may have changed, but the Arch ZFS+DKMS packages were a bit flaky and required some manual modification just to boot (should've taken this as a warning!).

Granted, it was my fault entirely for being a bit too enthusiastic with ZFS on Arch. To be honest, if I were to use it again, it would be on FreeBSD. Not Linux. I recognize it's fine for other people, but in my use case it wasn't.

Re: Bcachefs: “the COW filesystem for Linux that won't eat your data”

#78

Post by [deleted] »

slavapestov wrote: post #58

Here's a funny story. At one point bcache development was funded by a startup (which I won't name here). They were using it as the local storage layer of a distributed storage product. I worked there for a year in 2014. Apparently they were not aware of the fact that bcache was a) GPLd code, or b) developed before the company existed, first as a hobby project and then at Google. After a couple of years, they noticed…

[deleted]

Re: Bcachefs: “the COW filesystem for Linux that won't eat your data”

#79

Post by lomnakkus »

_ecqc wrote: post #25
zanny wrote: post #21

I've been using btrfs since about 2011, and I've stopped using ext4 / xfs / zfs everwhere since about 2014. From 2012-2014 it was mostly breakage every other month. From 2014-2016, it was semi-annual issues. For the last ~18 months I have had ~30 machines running btrfs with no issues, some servers, some personal computers. The release notes are boring, the bugs are boring, and to me its definitely in a state I would…

Are you running BTRFS with its built in RAID? That's been the biggest blocker for me. There have been numerous RAID bugs that have caused data-loss and I believe at least one of them is still unpatched.

My main issue isn't actually a single thing that's wrong -- it's the completely and utterly haphazard way many of the features in btrfs have been "designed"[1]. Some of these problem they've had seem, to me at least, to be a fundamental lack of a coherent design. That does not bode well for stability, even 10(?) years after its first version.

bcachefs seems to have a much more coherent design.

[1] "Oh, yeah, I don't know how to handle this code path yet, let's stick a BUG_ON in there! I'm sure we'll figure something out later."

Re: Bcachefs: “the COW filesystem for Linux that won't eat your data”

#80

Post by lomnakkus »

Zancarius wrote: post #77
tylerjd wrote: post #60

Earlier quoted context omitted.

ZFS is not GPL? I dunno, I like feeling safe when I do kernel upgrades knowing even if for whatever reason my ZFS module doesn't compile/work under the new kernel, I won't be left without a root FS. I have been running BTRFS for 5 years with very little issues, and enjoy not having to compile a new dkms module with every kernel.

ZFS with DKMS is a disaster, at least in my experience. Honestly, I can't recommend ZoL unless you're running a distro with relatively stable kernel releases that don't change substantially or that happens to be supported by ZoL with binary packages. ZoL on Arch was... trying at times. It worked great, but my paranoia meant that I ended up adding the kernel to IgnorePkg to force manual kernel updates (mostly for my o…

Your problem was running it on a distro the didn't support it. (As you acknowledge.)

So far it's been running absolutely great for me on several Ubuntu installs.

Post reply on HN