Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How to write a filesystem in 50 lines of code

blog.ksplice.com

Post reply on HN
Print view View original

21–30 of 31 posts

Re: How to write a filesystem in 50 lines of code

#21

Post by storborg »

silentbicycle wrote: post #18
TallGuyShort wrote: post #13

Earlier quoted context omitted.

Although I think they're overdone, I think "x in y lines" is avery relevant to the hacker community, while "x things about y" isn't always. Doing "x in y lines" generally implies an unusually elegant solution, that could be of great use.

It's such an easy metric to game, though ("The first line is 'import library_that_does_everything'") - It's a proxy for succinctness, at best. That said, "[something non-trivial] in only 250-500 LOC" is often more interesting than "...in only 5 LOC", because generally anything that small is just gluing together libraries. There are exceptions, of course - I've seen impressive stuff in just a few lines of J/K/APL.

In this case, the tool that he built (RouteFS) is actually a really clever abstraction above two other popular libraries (Routes and FUSE).

I agree it would be neat if he talked more about RouteFS itself, but it is a neat idea, and only about 200 lines.

Re: How to write a filesystem in 50 lines of code

#22

Post by [deleted] »

lisper wrote: post #14
price wrote: post #10

Earlier quoted context omitted.

Your data is fine -- it's hidden while something else is mounted on top of the folder. Rebooting will fix it. But you should be able to just unmount the filesystem, which will also fix it. Does 'mount' list something mounted there? Does 'df /path/to/folder/' reflect the FUSE filesystem rather than the underlying filesystem? The input/output error sounds like the FUSE filesystem being mounted indeed, and just being br…

> Your data is fine -- it's hidden while something else is mounted on top of the folder. I thought that was the case. But the truth is I don't really understand how mount points work under the hood. Where is the mount point information actually stored? I'm guessing it's in kernel memory somewhere, but I don't really know, and Googling hasn't been much help. Is there an article somewhere that explains this? > Rebootin…

[deleted]

Re: How to write a filesystem in 50 lines of code

#23

Post by barrkel »

price wrote: post #17
barrkel wrote: post #11

Earlier quoted context omitted.

Applications of the same idea have been around for some time, e.g. Plan 9's file system (e.g. GUI elements are part of the FS, , bash's /dev/tcp/ / etc., and indeed /proc's file system seen in Linux and, in a limited way, in Solaris. Seems like no Linux app framework can be complete without reinventing its own virtual file system, with various syntaxes for paths to e.g. network shares but that are inaccessible when u…

The point is that a FUSE filesystem is available from the command line and anywhere else, because it is an actual filesystem. RouteFS is a way of taking any virtual filesystem-like tree that you might find useful and making it available to the entire system as a normal filesystem, just as easily as you could describe the tree in any other form.

I know all that. I was bitching about how it seems like every layer of the software cake on Linux likes to design its own virtual file system.

Re: How to write a filesystem in 50 lines of code

#25

Post by snprbob86 »

djacobs wrote: post #24

Intriguing. I hadn't thought of this take on filesystems before. I'm a big fan of text-based config files but am stuck on a Windows machine at work. I wonder, would it be possible to map the registry to a virtual filesystem I could access from Explorer?

One of the many reasons to prefer PowerShell over CMD: http://powershell.com/cs/blogs/ebook/archive/2009/03/30/chap...

Re: How to write a filesystem in 50 lines of code

#26

Post by snprbob86 »

I know of a few professional console games that where successfully debugged by embedded telnet servers: it is incredibly useful to be able to cd and ls around in your scene graph. I believe the ones I saw implemented their own shells and toolset, but I bet you could get pretty clever with debug builds that expose a decent set of unix tools across a remote shell.

Re: How to write a filesystem in 50 lines of code

#27

Post by djacobs »

snprbob86 wrote: post #25
djacobs wrote: post #24

Intriguing. I hadn't thought of this take on filesystems before. I'm a big fan of text-based config files but am stuck on a Windows machine at work. I wonder, would it be possible to map the registry to a virtual filesystem I could access from Explorer?

One of the many reasons to prefer PowerShell over CMD: http://powershell.com/cs/blogs/ebook/archive/2009/03/30/chap...

Nice. I'm not too into DOS, but this definitely is an easier solution than creating a VFS for the job. Thanks for the pointer.

Re: How to write a filesystem in 50 lines of code

#28

Post by snprbob86 »

djacobs wrote: post #27
snprbob86 wrote: post #25

Earlier quoted context omitted.

One of the many reasons to prefer PowerShell over CMD: http://powershell.com/cs/blogs/ebook/archive/2009/03/30/chap...

Nice. I'm not too into DOS, but this definitely is an easier solution than creating a VFS for the job. Thanks for the pointer.

DOS != CMD != PowerShell

I used to be a hard-core Microsoft/Windows developer; in fact, I used to work for Microsoft! I was stuck in the Visual Studio sandbox and addicted to graphical tools. It has been a very slow transition, but I'm now addicted to my shell. If you are stuck on Windows, you should force yourself to learn and use PowerShell. And at home, you should install a Unix and force yourself to learn Bash. You'll thank me later.

Re: How to write a filesystem in 50 lines of code

#29

Post by ydant »

It's an interesting project. I grabbed it and wrote a quick Flickr FS. It seems that it's a little limited for really laggy webservices in that you specify the type of an entry (directory, folder) by the type of data you return, and it calls the same method for readdir, getattr, and read, with no way to differentiate in your code.

In order to specify the type of an entry, you have to return the data for that entry. So a sub-folder has to return an array. There's no way to differentiate between, say, an ls on the parent folder or on the sub folder (e.g. ls / and ls /foo both return the method mapped to /foo), so you have to query all of the subfolders' contents at once AND cache it so it doesn't have to be re-queried when the user wants to look at the sub-folder.

Hopefully I'm overlooking something, but the source is pretty straightforward. The good part is it'd be easy to modify to ask for types separately. Actually just passing in another argument indicating what mode it's in would help.

Re: How to write a filesystem in 50 lines of code

#30

Post by ebroder »

ydant wrote: post #29

It's an interesting project. I grabbed it and wrote a quick Flickr FS. It seems that it's a little limited for really laggy webservices in that you specify the type of an entry (directory, folder) by the type of data you return, and it calls the same method for readdir, getattr, and read, with no way to differentiate in your code. In order to specify the type of an entry, you have to return the data for that entry. S…

Yeah, that definitely can be a weakness of RouteFS's style.

My target application was things like automounters, or the low-latency database querying sort of thing I mention in the actual blog post. Since I wanted to be able to have the filesystem structure change as it was accessed, I decided to make any sort of caching entirely an application-layer problem, not a RouteFS-layer problem.

I think it would be possible to extend RouteFS to handle this sort of case more gracefully. One option in particular might be to take advantage of python-fuse's stateful I/O feature (which lets you associate a Python object with open file descriptors in your filesystem [1]) so that reads from the same file don't result in the same lookup over and over again, although this certainly doesn't help for directories.

But in any case, I'd certainly love to see ideas for extending RouteFS to make it easier to make it more performant. Submissions in the form of patches are always excellent, but even suggestions for API changes would be welcome - feel free to open an issue on Github either way (http://github.com/ebroder/python-routefs/issues).

[1] See "Filehandles can be objects if you want" in http://fuse.cvs.sourceforge.net/viewvc/fuse/python/README.ne... for more information

Post reply on HN