Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Muxfs – a mirroring, checksumming, and self-healing filesystem layer for OpenBSD

sdadams.org

Post reply on HN
Print view View original

21–30 of 65 posts

Re: Muxfs – a mirroring, checksumming, and self-healing filesystem layer for OpenBSD

#21

Post by masklinn »

arghwhat wrote: post #13
swinglock wrote: post #11

Earlier quoted context omitted.

Non ZFS filesystems are overly simplified, ignoring the problems they ought to be solving.

ZFS is not the simplest solution to the problems that "ought to be solved", and the implementation can be rather annoying - lacking support for hardware configuration changes, using its own cache system sidestepping the one in the kernel, and generally not fitting in with normal filesystem paradigms. And that's not even addressing that incremental sends - a huge feature - was (is?) broken due to holebirth, making it…

> ZFS is not the simplest solution to the problems that "ought to be solved"

What are simpler solutions to the problems that ought to be solved?

Re: Muxfs – a mirroring, checksumming, and self-healing filesystem layer for OpenBSD

#22

Post by nix23 »

anthk wrote: post #19
nix23 wrote: post #12

Earlier quoted context omitted.

Too sad, because the partitioning of OpenBSD is why i don't use it, with ZFS you could just do a dataset throw x^w,nosuid etc on them and give them a quota, with ffs one can bet that you run out of space (earlier or later), in one of the partitions (Workstation NOT Server).

I doubt it. Even for ports you can still symlink /usr/ports to $HOME/ports, for Scummvm with --enable-all-engines or Eduke32 (Build/GPLv2 license clash, can't be shared as a binary). /usr/local is not small at all by default.

No post body was provided.

Re: Muxfs – a mirroring, checksumming, and self-healing filesystem layer for OpenBSD

#23

Post by arghwhat »

masklinn wrote: post #21
arghwhat wrote: post #13

Earlier quoted context omitted.

ZFS is not the simplest solution to the problems that "ought to be solved", and the implementation can be rather annoying - lacking support for hardware configuration changes, using its own cache system sidestepping the one in the kernel, and generally not fitting in with normal filesystem paradigms. And that's not even addressing that incremental sends - a huge feature - was (is?) broken due to holebirth, making it…

> ZFS is not the simplest solution to the problems that "ought to be solved" What are simpler solutions to the problems that ought to be solved?

A CoW filesystem itself is not much more complicated than a plain filesystem. Or maybe a CoW softraid, with the filesystem existing at a different layer.

ZFS has countless bonus tunables, several types of caching distinct from the kernel VFS cache, its own write logs and special devices, multiple levels of topology (datasets in a pool consisting of vdevs consisting of drives), deduplication, compression, etc.

It is also not at all user friendly. When set up right (and no changing your mind on setup), and when fed enough resources, it does it's job well, but simple or elegant cannot describe it.

Re: Muxfs – a mirroring, checksumming, and self-healing filesystem layer for OpenBSD

#24

Post by arghwhat »

swinglock wrote: post #15
arghwhat wrote: post #13

Earlier quoted context omitted.

ZFS is not the simplest solution to the problems that "ought to be solved", and the implementation can be rather annoying - lacking support for hardware configuration changes, using its own cache system sidestepping the one in the kernel, and generally not fitting in with normal filesystem paradigms. And that's not even addressing that incremental sends - a huge feature - was (is?) broken due to holebirth, making it…

Some of them are grounded in Linux politics. I have hope in bcachefs but best not rush, we've all seen btrfs.

ZFS was never any prettier on BSD either, so wouldn't blame Linux in that.

But yes, bcachefs is somewhat interesting. Or maybe btrfs manages to clean up their act one day.

Re: Muxfs – a mirroring, checksumming, and self-healing filesystem layer for OpenBSD

#25

Post by adrian_b »

ciprian_craciun wrote: post #9

I have only two issues with the `muxfs` implementation as it stands: (1) (And the largest problem) is that it requires stable inodes in order to tie the checksums with the actual files. This means (and it's already stated in the article) you can't copy / move / overwrite any of the underlying files without losing the checksums. (Basically it also removes the possibility of accessing one of the mirrors via NFS, FUSE,…

I am also using checksums to detect bit-rot, but in order to tie them to the files they are stored in extended attributes of the files. Thus they do not depend on the inode numbers.

OpenBSD also supports extended file attributes, so using them should be possible.

Using extended attributes on Linux or FreeBSD requires a few precautions, because there are still various copying/movement/archiving CLI commands or GUI applications that ignore the extended attributes and also some file systems that do not implement extended attributes, e.g. tmpfs on Linux (which supports only certain kinds of system extended attributes, not those defined by the users) or all not extremely new versions of NFS (only NFSv4 in Linux 5.9 or newer supports xattr, unlike samba, which has supported them for decades, mapping them correctly between different file systems, e.g. XFS on Linux to UFS on FreeBSD), so copying a file via those file systems would lose silently the extended attributes of the files.

The extended file attributes have been introduced in 1989, in HPFS for OS/2 version 1.2, and they have been brought to UNIX in XFS, in 1993.

30 years later, it is annoying to see that there are still some programs which pretend to make file copies or file archives, but which can lose the extended file attributes, without any warnings or errors.

Re: Muxfs – a mirroring, checksumming, and self-healing filesystem layer for OpenBSD

#27

Post by fbhabbed »

arghwhat wrote: post #24
swinglock wrote: post #15

Earlier quoted context omitted.

Some of them are grounded in Linux politics. I have hope in bcachefs but best not rush, we've all seen btrfs.

ZFS was never any prettier on BSD either, so wouldn't blame Linux in that. But yes, bcachefs is somewhat interesting. Or maybe btrfs manages to clean up their act one day.

What's wrong with btrfs?

Re: Muxfs – a mirroring, checksumming, and self-healing filesystem layer for OpenBSD

#28

Post by sdadams »

ciprian_craciun wrote: post #9

I have only two issues with the `muxfs` implementation as it stands: (1) (And the largest problem) is that it requires stable inodes in order to tie the checksums with the actual files. This means (and it's already stated in the article) you can't copy / move / overwrite any of the underlying files without losing the checksums. (Basically it also removes the possibility of accessing one of the mirrors via NFS, FUSE,…

(1) I believe that stable inode numbers are possible with FUSE but they must be implemented by the FUSE driver. Mirroring over a network is not the intended use case. The mirrors exist to provide data redundancy for use by the muxfs driver. If you need to copy the data to a new location there is the muxfs sync command.

(2) muxfs uses sequence numbers to count the write operations performed on each mirror. Upon failure to mount due to the mirrors being out of sync a report is printed comparing the first mirror with the first non-matching mirror, and this includes their sequence numbers.

Re: Muxfs – a mirroring, checksumming, and self-healing filesystem layer for OpenBSD

#29

Post by sdadams »

zasdffaa wrote: post #17
defrost wrote: post #2

Nice concept, and having skimmed it's worth noting: > muxfs needs you! > No filesystem can be considered stable without thorough testing and muxfs is no exception. > Even if I had tested muxfs enough to call it stable it still would not be responsible to expect you to simply take my word for it. It is for this reason that I do not intend to release a version 1.0 until there are sufficient citations that I can make to…

These requirements of much testing of inherently multithreaded code with a lot riding on not destroying user data suggests a model checker would be an essential piece, perhaps someone more knowledgeable can opine.

OpenBSD's FUSE does not implement multithreading.

Re: Muxfs – a mirroring, checksumming, and self-healing filesystem layer for OpenBSD

#30

Post by klysm »

arghwhat wrote: post #23
masklinn wrote: post #21

Earlier quoted context omitted.

> ZFS is not the simplest solution to the problems that "ought to be solved" What are simpler solutions to the problems that ought to be solved?

A CoW filesystem itself is not much more complicated than a plain filesystem. Or maybe a CoW softraid, with the filesystem existing at a different layer. ZFS has countless bonus tunables, several types of caching distinct from the kernel VFS cache, its own write logs and special devices, multiple levels of topology (datasets in a pool consisting of vdevs consisting of drives), deduplication, compression, etc. It is a…

I completely agree with that assessment. The core ideas behind how the CoW work are elegant, but the implementation is anything but elegant.
Post reply on HN