Show HN: MergerFS – A Featureful Union Filesystem
1–10 of 10 posts
Re: Show HN: MergerFS – A Featureful Union Filesystem
#2[dead]
Re: Show HN: MergerFS – A Featureful Union Filesystem
#3Is this really a 'Show HN'? Anyway, I'm not affiliated with the maintainer either, but if it's of use to anyone reading I do maintain a Docker image for mergerfs:
https://github.com/OJFord/docker-mergerfs/
Re: Show HN: MergerFS – A Featureful Union Filesystem
#4The "Why?" is hard to find, but I think this answer to "Why use mergerfs over unionfs" is informative:
> UnionFS is more like aufs than mergerfs in that it offers overlay / CoW features. If you're just looking to create a union of drives and want flexibility in file/directory placement, then mergerfs offers that, whereas unionfs is more for overlaying RW filesystems over RO ones.
Also:
> What should mergerfs NOT be used for?
1. databases: Even if the database stored data in separate files (mergerfs wouldn't offer much otherwise) the higher latency of the indirection will kill performance. If it is a lightly used SQLITE database then it may be fine but you'll need to test.
2. VM images: For the same reasons as databases. VM images are accessed very aggressively and mergerfs will introduce too much latency (if it works at all).
3. As replacement for RAID: mergerfs is just for pooling branches. If you need that kind of device performance aggregation or high availability you should stick with RAID.
Re: Show HN: MergerFS – A Featureful Union Filesystem
#5Cool tool, but recommend reading the policies [0] section carefully, it did catch me off guard the first time I tried using it.
E.g. say you have /a/foo.txt and /b/foo.txt, and you run mergerfs /a:/b /merged
Now if you print or modify /merged/foo.txt, it would access /a/foo.txt because the default policy for file access is ff (first-found, as defined during the mount).
However, if you run "rm /merged/foo.txt", it would delete it from both /a and /b, because the default policy for unlink call would be "epall" (existing path, apply to all).
[0] https://github.com/trapexit/mergerfs#policy-descriptions
Re: Show HN: MergerFS – A Featureful Union Filesystem
#6If you find this interesting then you might also be interested in SnapRAID: https://www.snapraid.it/
> SnapRAID is a backup program for disk arrays. It stores parity information of your data and it recovers from up to six disk failures.
> SnapRAID is mainly targeted for a home media center, with a lot of big files that rarely change.
> If the failed disks are too many to allow a recovery, you lose the data only on the failed disks. All the data in the other disks is safe.
I'm not affiliated with any of the projects.
Re: Show HN: MergerFS – A Featureful Union Filesystem
#7If you find this interesting then you might also be interested in SnapRAID: https://www.snapraid.it/ > SnapRAID is a backup program for disk arrays. It stores parity information of your data and it recovers from up to six disk failures. > SnapRAID is mainly targeted for a home media center, with a lot of big files that rarely change. > If the failed disks are too many to allow a recovery, you lose the data only on th…
I’ve used MergerFS and SnapRAID for my media server for about two years. Together they’re really flexible and I’ve been able to drop extra drives into the array as I’ve needed the storage. On that capability alone I highly recommend using them.
I haven’t had a failure yet but I imagine I wouldn’t be in trouble unless I lost a significant portion of my parity drives at the same time.
Re: Show HN: MergerFS – A Featureful Union Filesystem
#8Cool tool, but recommend reading the policies [0] section carefully, it did catch me off guard the first time I tried using it. E.g. say you have /a/foo.txt and /b/foo.txt, and you run mergerfs /a:/b /merged Now if you print or modify /merged/foo.txt, it would access /a/foo.txt because the default policy for file access is ff (first-found, as defined during the mount). However, if you run "rm /merged/foo.txt", it wou…
The alternative behaviour seems undesirable too right? If you attempted to delete foo.txt in the merged filesystem and instead of deleting the file, it just replaced the contents (with b)
I think the behaviour makes sense. Otherwise deleting a shadowed file either requires extra state (to track whether to show the shadowed file after a deletion), or loses some of the implicit assumptions people make for filesystems (after I successfully rm, the file is gone).
Re: Show HN: MergerFS – A Featureful Union Filesystem
#9Cool tool, but recommend reading the policies [0] section carefully, it did catch me off guard the first time I tried using it. E.g. say you have /a/foo.txt and /b/foo.txt, and you run mergerfs /a:/b /merged Now if you print or modify /merged/foo.txt, it would access /a/foo.txt because the default policy for file access is ff (first-found, as defined during the mount). However, if you run "rm /merged/foo.txt", it wou…
The alternative behaviour seems undesirable too right? If you attempted to delete foo.txt in the merged filesystem and instead of deleting the file, it just replaced the contents (with b) I think the behaviour makes sense. Otherwise deleting a shadowed file either requires extra state (to track whether to show the shadowed file after a deletion), or loses some of the implicit assumptions people make for filesystems (…
Ah agree, your example makes sense!
I guess it ultimately depends on the usecase, I just gave the above as an easy to understand example. In my case I wanted something that merely merged multiple distinct directories as a unified view (just for convenience). Ideally they wouldn't be overlapping, but I was still a bit paranoid about this behaviour.
In addition had another confusion, the default policy for creating new files/directories was epmfs (create at the "most free space" partition). So as a result, when I created new directories/files, it seemed to do that seemingly at random.
Of course my bad, should have been a bit more careful, the man page/readme seem to explain it well :)
Re: Show HN: MergerFS – A Featureful Union Filesystem
#10Cool tool, but recommend reading the policies [0] section carefully, it did catch me off guard the first time I tried using it. E.g. say you have /a/foo.txt and /b/foo.txt, and you run mergerfs /a:/b /merged Now if you print or modify /merged/foo.txt, it would access /a/foo.txt because the default policy for file access is ff (first-found, as defined during the mount). However, if you run "rm /merged/foo.txt", it wou…
The alternative behaviour seems undesirable too right? If you attempted to delete foo.txt in the merged filesystem and instead of deleting the file, it just replaced the contents (with b) I think the behaviour makes sense. Otherwise deleting a shadowed file either requires extra state (to track whether to show the shadowed file after a deletion), or loses some of the implicit assumptions people make for filesystems (…
> loses some of the implicit assumptions people make for filesystems
This is sort of incorrect though. Unless your file system supports transactions (which I assume this one does not), this should never be your expectation.