HTTP is basically a filesystem protocol that supports magical files -- not too unlike sharing named pipes over SMB. Not too unlike what it'd be like if one could open(2) AF_LOCAL sockets on Unix/POSIX systems instead of having to connect(2) to them -- if that had been so in 1982 in BSD, it would be true now, NFS would have supported the same, etc.
Webfs: A Filesystem Built on Top of the Web
41–50 of 59 posts
Re: Webfs: A Filesystem Built on Top of the Web
#42Earlier quoted context omitted.
I'm going to assume you're familiar with Plan9OS's WebFS and 9P Protcol? https://en.m.wikipedia.org/wiki/9P_(protoco) I've been kicking around similar ideas (nowhere near implemented) for a while: https://old.reddit.com/r/dredmorbius/comments/6bgowu/what_if...
I was not aware of Plan 9's WebFS. It looks like it presents websites in the local file system. IPFS can do something very similar with its content. I have used that before. The name here comes from using web resources (referred to by url) as building blocks for a file system. I guess "Web for FS" rather than "Web as FS".
The notion of remote services accessed via local filesystem dynamics is pretty well established. Among implementations:
- NFS, particularly with the Solaris-originated concept of automounts over a /net mountpoint.
- Various virtual filesystems. Midnight Commander ("mc") offers several of these, including archive formats (tar, cpio, afio, rpm, deb) and remote (FTP, SSH).
- SMB/CFS/Samba
- Various FUSE filesystems, including again ssh, ftp, and others. These generally require specifying in advance specific mountpoints.
The notion of on-demand access to remote resources over protocols (e.g., http/https, or others), under filesystem dynamics, is interesting -- you can use any general tool, utility, or application for access, mediated through the filesystem by way of drivers, rather than a specific application (e.g., Web browser, FTP client, etc.)
There are numerous issues. In particular, applications tend not to respond well to remote resources disappearing, changing, or failing to return from change requests -- NFS's behaviour with nonresponsive remote hosts is ... notorious.
Consistency, availability, and partition resistance (CAP) are long-standing concerns, and there's no way to solve for all three. I'd add latency as another major consideration.
https://en.wikipedia.org/wiki/CAP_theorem
The general notion of managing and tracking changes locally, and pushing them to remote, has merits. I'm not aware of a "gitfs" ... though of course, one does exist, TIL: https://www.presslabs.com/code/gitfs/ The notion of using git (or other versioning system) as a mediator for remote/local revisioned access seems to have merits. Obviously not viable for very-high-variability systems, but adequate for many occasionally-modified resources.
I'm not sure if you're looking at using your WebFS itself as a publishing mechanism, though in general I think I'd recommend against doing this. For small-n peer-to-peer distribution that's probably workable, but for large-scale provisioning-and-request systems, relying on HTTP or other established transports is likely more sensible.
One area I've recognised as being particularly fraught is the whole notion of security and privacy. Providing unfiltered local access to remote resources which may change arbitrarily is a great way for allowing malware onto local systems -- your transport layer should probably implement some level of security and mounts deny direct execution of content. The fact that remote content could be copied to an executable mountpoint remains, and would make numerous attacks possible.
Similarly: access, update, write, and/or publishing actions all leak considerable information which could be of concern to specific users or organisations. Hash-based indexing (already addressed in this thread) being only one of several such vectors.
Re: Webfs: A Filesystem Built on Top of the Web
#43Remember WebDAV? It was a similar concept, but never really found its footing and most of the implementations were pretty shaky. I always thought it was a good idea though. https://en.wikipedia.org/wiki/WebDAV
Ah yes... I implemented Sardine [1] 10 years ago! At the time, I worked for a large porn company and we couldn't host stuff 'in the cloud' because they didn't allow porn there. We invested a ton of money into an Isilon NAS to store our image/video content and the best way to get stuff off it over HTTP was via webdav. Unfortunately, there wasn't a good Java client. So, I built a simple proxy that would accept regular…
I implemented tve same for ruby. With monkey patching you write normal code for writing files, but use the WebDAV protcol to write files if files had names starting with http.
Re: Webfs: A Filesystem Built on Top of the Web
#44Hey everyone, I'm the author of WebFS. Happy to answer any questions.
Name is dangerously close to WebNFS.
Re: Webfs: A Filesystem Built on Top of the Web
#45Hey everyone, I'm the author of WebFS. Happy to answer any questions.
This looks really cool.
> If you have ever thought "x can probably be used as a file system"
...you might also want to take a look at Storage Combinators[1]. Not quite the same problem space, but abstracting away a bit from both concrete filesystems and other storage mechanisms to get to a composable abstraction.
Note: I am the primary author, and also taking a good look at WebFS for further inspiration... :-)
[1] https://2019.splashcon.org/details/splash-2019-Onward-papers...
Re: Webfs: A Filesystem Built on Top of the Web
#46Remember WebDAV? It was a similar concept, but never really found its footing and most of the implementations were pretty shaky. I always thought it was a good idea though. https://en.wikipedia.org/wiki/WebDAV
I use it every day on Android, Windows and Linux. It found its footing.
Re: Webfs: A Filesystem Built on Top of the Web
#47Earlier quoted context omitted.
True. But Tor onions are servers.
Not in any sense that's relevant here. They don't host content.
I have no clue where you're coming from.
What sort of content do you say Tor onions can't host?
"Tor onion" just means that a server is (ideally) only reachable as an onion URL, which is only accessible via the Tor network. There is the limitation that Tor only handles TCP. Otherwise, one can route anything over Tor. In my experience, that includes HTTP(S), FTP, Tahoe-LAFS, SSH, RDP, Mumble, OpenVPN and tinc. And others, if I spent more time remembering what I've played with.
Re: Webfs: A Filesystem Built on Top of the Web
#48Re: Webfs: A Filesystem Built on Top of the Web
#49Earlier quoted context omitted.
True. But Tor onions are servers.
I've never heard of an onion service called just "an onion" before. I don't know enough to say if that's wrong, just that I was confused.
Sorry to be confusing. It is commonly used, in some Tor communities. But yes, "onion service" is clearer.
Re: Webfs: A Filesystem Built on Top of the Web
#50Hey everyone, I'm the author of WebFS. Happy to answer any questions.
You will probably be interested in Peergos [0][1], which at it's lowest level is an encrypted global filesystem also built on IPFS and also only using the block api.