Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Userspace FUSE for macOS

fuse-t.org

Post reply on HN
Print view View original

11–20 of 137 posts

Re: Userspace FUSE for macOS

#11

Post by EdSchouten »

EdSchouten wrote: post #5

What a coincidence! For a project that I maintain (Buildbarn, a distributed build cluster for Bazel) I recently generalized all of the FUSE code I had into a generic VFS that can both be exposed over FUSE and NFSv4. The intent was the same: to provide a better out of the box experience on macOS. Here's a design doc I wrote on that change. Slight warning that it's written with some Buildbarn knowledge in mind. https:/…

Replying to my own comment:

I do wonder how this library deals with some of the fundamental differences beween FUSE and NFSv4. For example, with FUSE the kernel and server share intimate knowledge on which part of the file system lives in the kernel's inode cache. Only when the kernel issues a FORGET call, may the server drop information corresponding to a given nodeid.

As NFSv4 is designed to be stateless, servers may need to be able to process requests containing arbitrary file handles, regardless of how long ago they were returned as part of some prior request. You therefore see that in-kernel implementations of NFS servers rely on special file system methods to resolve objects by file handle, in addition to being able to resolve by path.

This means that if you implement FUSE on top of NFSv4, you will most likely not be able to purge any state. Your FUSE file system's FORGET method will probably never be called. This means that memory usage of fuse-t will most likely just keep on growing as time progresses? Or it announces itself as using file handle mode FH4_VOLATILE_*, but UNIX-like NFSv4 clients hardly ever know how to deal with that.

Re: Userspace FUSE for macOS

#14

Post by boulos »

EdSchouten wrote: post #11
EdSchouten wrote: post #5

What a coincidence! For a project that I maintain (Buildbarn, a distributed build cluster for Bazel) I recently generalized all of the FUSE code I had into a generic VFS that can both be exposed over FUSE and NFSv4. The intent was the same: to provide a better out of the box experience on macOS. Here's a design doc I wrote on that change. Slight warning that it's written with some Buildbarn knowledge in mind. https:/…

Replying to my own comment: I do wonder how this library deals with some of the fundamental differences beween FUSE and NFSv4. For example, with FUSE the kernel and server share intimate knowledge on which part of the file system lives in the kernel's inode cache. Only when the kernel issues a FORGET call, may the server drop information corresponding to a given nodeid. As NFSv4 is designed to be stateless, servers m…

Nit: NFSv3 is stateless, V4 isn't (sessionid and clientid permit stateful handling of locks and shares).

I would assume this person is just doing an in-memory "NFS server" that would keep all of that state around. So it's more like a FUSE-compatible layer that speaks NFSv4 as a "front end" (since NFS clients are better than "ha ha, surprise! This FUSE backend is networked and can fail on weird ways").

I'm not sure how they chose to translate things like DELEGATE, SEQUENCE, and so on (probably just reply with an error). But for basic OPEN, READ, WRITE, etc. it's all fairly straightforward.

I had hoped they used nfs-ganesha for the NFS server / frontend, but the attributions file suggests they probably rolled their own (and thus is going to be full of bugs and immature for quite some time).

Re: Userspace FUSE for macOS

#15

Post by mdaniel »

HHad3 wrote: post #9

This is great! I hope that this project enables those many FUSE-related applications to return to Homebrew since an open-source FUSE provider is now available again. I am curious though why a NFSv4 server was chosen over wrapping Apple's File Provider API [1], which seems to be the native method for providing virtual file systems from user space on macOS since macOS 11.5. After glancing over the API I guess it's beca…

It seems based on the associated WWDC video (https://developer.apple.com/videos/play/wwdc2021/10182/) that API appears to be for making your own Dropbox/GDrive/OneDrive client, and not "present this totally fake filesystem" scenario

Re: Userspace FUSE for macOS

#16

Post by mdaniel »

tambourine_man wrote: post #12

macOS really needs a decent FUSE implementation. That and some sort of container story. Such a shame that it's not really open source. The idea of using NFSv4 is awesome though.

> That and some sort of container story.

While off-topic for this thread, I don't think that's going to do what you expect. If you snapped your fingers and XNU suddenly had cgroups and other namespace trickery required to make containers operate, you'd still have the grave problem of "containers are not virtual machines" and thus an XNU container would only run Darwin binaries, so you're back to the old days of "run one exe locally, run another in prod"

Re: Userspace FUSE for macOS

#17

Post by EdSchouten »

boulos wrote: post #14
EdSchouten wrote: post #11

Earlier quoted context omitted.

Replying to my own comment: I do wonder how this library deals with some of the fundamental differences beween FUSE and NFSv4. For example, with FUSE the kernel and server share intimate knowledge on which part of the file system lives in the kernel's inode cache. Only when the kernel issues a FORGET call, may the server drop information corresponding to a given nodeid. As NFSv4 is designed to be stateless, servers m…

Nit: NFSv3 is stateless, V4 isn't (sessionid and clientid permit stateful handling of locks and shares). I would assume this person is just doing an in-memory "NFS server" that would keep all of that state around. So it's more like a FUSE-compatible layer that speaks NFSv4 as a "front end" (since NFS clients are better than "ha ha, surprise! This FUSE backend is networked and can fail on weird ways"). I'm not sure ho…

With it being stateless, I was referring to just the part where I wanted to point out the difference: file handles are stateless, while FUSE's equivalent (nodeids) are stateful.

Regardless of whether it's an in-memory or persistent solution, the problem remains: fuse-t has little choice but to leak resources of the underlying FUSE file system, as there is no valid point in time in which you can issue FORGET operations. This means that any file system with some form of churn rate will leak memory.

Also note that delegation is effectively optional. If a server simply always replies with OPEN_DELEGATE_NONE, the client has to interact with the file as if it's stored remotely.

There is no need to implement SEQUENCE, by the way. macOS implements NFSv4.0, while SEQUENCE is part of NFSv4.1 and later.

Re: Userspace FUSE for macOS

#18

Post by [deleted] »

rgovostes wrote: post #4

Unfortunately, like macFUSE, this is not really open source. The license appears to be BSD-like for non-commercial use only. Also like macFUSE, the project uses GitHub but the source code is not available. https://github.com/macos-fuse-t/fuse-t/blob/main/License.txt

[deleted]
Post reply on HN