Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Userspace FUSE for macOS

fuse-t.org

Post reply on HN
Print view View original

51–60 of 137 posts

Re: Userspace FUSE for macOS

#51

Post by FullyFunctional »

concerned_ctzn wrote: post #38
jbverschoor wrote: post #28

Why does everybody use tcp ports instead of file sockets for local communication?

tcp ports are much more versatile, you can expose it to the outside world as opposed to unix sockets. Now exposing an nfs server to the outside can lead to interesting possibilities: for example you can implement a local fuse file system which can be mounted remotely through the NFS. It's highly not recommended at this moment because there's no authentication implemented

Is there any fundamental reason you couldn't do both? Presumably local sockets have less overhead (and I do prefer not to open ports unless needed).

Re: Userspace FUSE for macOS

#52

Post by EdSchouten »

concerned_ctzn wrote: post #48
EdSchouten wrote: post #17

Earlier quoted context omitted.

With it being stateless, I was referring to just the part where I wanted to point out the difference: file handles are stateless, while FUSE's equivalent (nodeids) are stateful. Regardless of whether it's an in-memory or persistent solution, the problem remains: fuse-t has little choice but to leak resources of the underlying FUSE file system, as there is no valid point in time in which you can issue FORGET operation…

I don't understand the comment about leaking memory, nfs file handles don't have to be persistent. Besides, how many FUSE filesystems implement FORGET? Anyway this is the reason I went with NFSv4. I started the project with NFS v3 but then discovered many limitations: being stateless, no named attributes, questionable locking support, etc. So Eventually I dropped it and re-implemented everything on NVSv4.

> I don't understand the comment about leaking memory, nfs file handles don't have to be persistent.

As long as the file the NFSv4 file handle refers to is still usable (i.e., linked into the file system), the NFSv4 file handle must remain usable. Note that this is not a universal requirement, but at least one that the macOS NFSv4 client enforces. It only implements FH4_PERSISTENT. This doesn't seem to be documented explicitly, but is somewhat revealed by this printf():

https://github.com/apple/darwin-xnu/blob/main/bsd/nfs/nfs_vf...

> Besides, how many FUSE filesystems implement FORGET?

Any file system that wants to remove files in the background (meaning: not by calling unlink() through the FUSE mount) must likely do proper refcounting on such files, and provide an implementation of FORGET.

For example, a file system that can give information on live football matches may want to remove files/directories belonging to matches that have already ended. In that case you want the node IDs to remain valid, but refer to files that have already been unlinked from the file system hierarchy. The FORGET operation allows you to determine when those files can be removed from the FUSE server's bookkeeping entirely.

Here is an implementation of FORGET that I wrote for Buildbarn:

https://github.com/buildbarn/bb-remote-execution/blob/master...

Re: Userspace FUSE for macOS

#53

Post by FullyFunctional »

boulos wrote: post #14
EdSchouten wrote: post #11

Earlier quoted context omitted.

Replying to my own comment: I do wonder how this library deals with some of the fundamental differences beween FUSE and NFSv4. For example, with FUSE the kernel and server share intimate knowledge on which part of the file system lives in the kernel's inode cache. Only when the kernel issues a FORGET call, may the server drop information corresponding to a given nodeid. As NFSv4 is designed to be stateless, servers m…

Nit: NFSv3 is stateless, V4 isn't (sessionid and clientid permit stateful handling of locks and shares). I would assume this person is just doing an in-memory "NFS server" that would keep all of that state around. So it's more like a FUSE-compatible layer that speaks NFSv4 as a "front end" (since NFS clients are better than "ha ha, surprise! This FUSE backend is networked and can fail on weird ways"). I'm not sure ho…

TIL about nfs-ganesha. I have yet to digest it all, but ignoring the wonderful usability of sshfs, it seems you should be able to get the functionality of sshfs with nfs-ganesha? Am I missing something?

Re: Userspace FUSE for macOS

#55

Post by eptcyka »

duped wrote: post #31
inetknght wrote: post #30

Earlier quoted context omitted.

> Why does everybody use tcp ports instead of file sockets for local communication? In my experience it's because Windows and mac developers aren't aware of local file sockets. Windows API, in particular, doesn't have a similar concept if I recall.

Windows has named pipes. MacOS has UDS and they're not terribly uncommon

Windows also supports UDS now.

Re: Userspace FUSE for macOS

#56

Post by concerned_ctzn »

EdSchouten wrote: post #52
concerned_ctzn wrote: post #48

Earlier quoted context omitted.

I don't understand the comment about leaking memory, nfs file handles don't have to be persistent. Besides, how many FUSE filesystems implement FORGET? Anyway this is the reason I went with NFSv4. I started the project with NFS v3 but then discovered many limitations: being stateless, no named attributes, questionable locking support, etc. So Eventually I dropped it and re-implemented everything on NVSv4.

> I don't understand the comment about leaking memory, nfs file handles don't have to be persistent. As long as the file the NFSv4 file handle refers to is still usable (i.e., linked into the file system), the NFSv4 file handle must remain usable. Note that this is not a universal requirement, but at least one that the macOS NFSv4 client enforces. It only implements FH4_PERSISTENT. This doesn't seem to be documented…

I'm not sure I understand you point, if an inode gets removed, then GETATTR or LOOKUP would fail as it should unless you're talking about open files. In the latter case the inode will get removed when the last open handle to the file is closed

Re: Userspace FUSE for macOS

#57

Post by tambourine_man »

alerighi wrote: post #40
tambourine_man wrote: post #12

macOS really needs a decent FUSE implementation. That and some sort of container story. Such a shame that it's not really open source. The idea of using NFSv4 is awesome though.

Awesome, to me it seems terrible, and something that can break anytime. You have two layers of translation, the actual FUSE filesystem implementation that uses the FUSE API to communicate with something that emulates a FUSE API in userspace to then talk via NFS to the kernel. NFS is broken in a lot of ways, especially with file locking, that could result in deadlocks. Really FUSE should be implemented in the kernel i…

You’re right, a proper userspace filesystem implementation is the preferable route. I was mostly impressed by the neet hack.

Re: Userspace FUSE for macOS

#58

Post by justsomeadvice0 »

EdSchouten wrote: post #5

What a coincidence! For a project that I maintain (Buildbarn, a distributed build cluster for Bazel) I recently generalized all of the FUSE code I had into a generic VFS that can both be exposed over FUSE and NFSv4. The intent was the same: to provide a better out of the box experience on macOS. Here's a design doc I wrote on that change. Slight warning that it's written with some Buildbarn knowledge in mind. https:/…

You might look at webdav instead, which has actual existing library implementations (unlike NFSv4), supports user/pass auth (in some cases you don't want wide-open services on loopback) and is based on HTTP; that would make the whole thing easier. Various projects offer a webdav combat layer, see cryptomator for one example: https://docs.cryptomator.org/en/latest/desktop/vault-mountin...

I went down the NFS road long ago - I somewhat (??) remember that coaxing macOS to mount_nfs on loopback is actually a PITA (I remember having to activate 127.0.0.* and mount 127.0.0.2).

Re: Userspace FUSE for macOS

#59

Post by Cyberdog »

tambourine_man wrote: post #47
duskwuff wrote: post #24

Earlier quoted context omitted.

You can install multiple versions of PHP and Node alongside each other just fine using Homebrew: brew install php@7.2 php@7.3 php@7.4 php@8.0 php@8.1 # and soon php@8.2 brew install node@18 node@16 node@14 node@12 node@10

Installing multiple versions is solved, changing the environment easily, not so much, I think

I'm not sure of the specifics with Homebrew, but with MacPorts, I have both PHP 7.4 and 8.1 running via FPM and serving sites rather trivially. The basics: Install both php74-fpm and php81-fpm, configure the former to put its socket at /var/run/php74-fpm.sock and the latter at /var/run/php81-fpm.sock, configure nginx's domain-specific config files to look for the FastCGI socket that the respective path, use MacPorts to load both daemons, and away you go. I imagine a similar approach would be possible with Homebrew.

Re: Userspace FUSE for macOS

#60

Post by klabb3 »

pstuart wrote: post #29
jbverschoor wrote: post #28

Why does everybody use tcp ports instead of file sockets for local communication?

2 guesses: ignorance, and an assumption that using tcp allows for seamless transition to another host.

I guess I'm ignorant then! When I looked up domain sockets and so on it turned out to be different APIs for different OS:s, and it's significantly nicer to rely on a single API surface from the std lib. Maybe it's a habit thing as well, but to me pipes are more esoteric and harder to find docs about than network sockets.
Post reply on HN