Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The File Filesystem (2021)

mgree.github.io

Post reply on HN
Print view View original

41–50 of 105 posts

Re: The File Filesystem (2021)

#41

Post by duped »

Groxx wrote: post #29
duped wrote: post #26

Earlier quoted context omitted.

Really what you'd like to see is a way to write the mount command for each file type (do one thing well) and another command to detect the file type and dispatch accordingly (probably similar to the `file` command), all in user space. The only thing standing in the way of this today is that MacOS doesn't expose a user space file system API. You can do this on Linux, Windows, and BSDs today. (No, file provider extensi…

Does https://osxfuse.github.io/ cover this? Or is there some fundamental issue? (beyond "it's not built in")

Well that requires a kext so it's a nonstarter, and fuse-t uses NFS which is extremely janky and unreliable on MacOS.

The fundamental issue is that macOS doesn't provide an API for this natively.

Re: The File Filesystem (2021)

#42

Post by ranger_danger »

Scaevolus wrote: post #36
paulgb wrote: post #28

This is really neat, but when I saw the headline I got excited that it was something I have been looking for / considering writing, and I figure the comments here would be a good place to ask if something like this exists: Is there a FUSE filesystem that runs in-memory (like tmpfs) while mounted, and then when dismounted it serializes to a single file on disk? The closest I can find are FUSE drivers that mount archiv…

Not purely in-memory, but something like https://github.com/jrwwallis/qcow2fuse maybe? It's clunky compared to OSX's DMGs, but if you squint it achieves similar ends. Otherwise you could achieve this with a tmpfs wrapped to serialize to a tarball (preserving symlinks) when unmounted.

Oh nice, I didn't even know that existed. I've been using qemu-nbd and parted by hand and it gets cumbersome, so this might help a lot. Thanks!

Re: The File Filesystem (2021)

#43

Post by [deleted] »

wegwerff wrote: post #8
yjftsjthsd-h wrote: post #4

I would gently suggest naming it filefs or something; ffs already means https://man.freebsd.org/cgi/man.cgi?ffs(7) That said - good idea/approach; seems like an excellent way to cleanly extend the unix approach to structured file formats:)

also https://en.wikipedia.org/wiki/Formatted_File_System and https://en.wikipedia.org/wiki/Flash_file_system and kind of https://en.wikipedia.org/wiki/Amiga_Fast_File_System Four file systems named ffs; ffs is popular for file systems

[deleted]

Re: The File Filesystem (2021)

#44

Post by lmm »

RetroTechie wrote: post #21

Useful enough that it should be an OS-level standard feature, imho. Unix-like OSes allow mounting disk images to explore their contents. But there's many more file formats where exploring files-inside-files is useful. Compressed archives, for one. Some file managers support those, but (imho) application-level is not the optimal layer to put this functionality. Could be implemented with a kind of driver-per-filetype.

This was a core design feature of reiserfsv4, but Linux ultimately refused to merge it, probably not helped by the whole murdering-his-wife thing.

Re: The File Filesystem (2021)

#45

Post by AgentME »

ranger_danger wrote: post #38
paulgb wrote: post #28

This is really neat, but when I saw the headline I got excited that it was something I have been looking for / considering writing, and I figure the comments here would be a good place to ask if something like this exists: Is there a FUSE filesystem that runs in-memory (like tmpfs) while mounted, and then when dismounted it serializes to a single file on disk? The closest I can find are FUSE drivers that mount archiv…

I can't think of anything _exactly_ like that, but I think you can get close by just copying some type of image file to /tmp and then moving it to disk when you're done after unmounting.

/tmp isn't stored in memory; it's usually a normal on-disk filesystem that's cleared regularly. You want /dev/shm instead, which is a purely in-memory filesystem on normal Linux systems.

Re: The File Filesystem (2021)

#46

Post by codetrotter »

AgentME wrote: post #45
ranger_danger wrote: post #38

Earlier quoted context omitted.

I can't think of anything _exactly_ like that, but I think you can get close by just copying some type of image file to /tmp and then moving it to disk when you're done after unmounting.

/tmp isn't stored in memory; it's usually a normal on-disk filesystem that's cleared regularly. You want /dev/shm instead, which is a purely in-memory filesystem on normal Linux systems.

> /tmp isn't stored in memory

It is if your system uses tmpfs for /tmp

https://en.wikipedia.org/wiki/Tmpfs

Re: The File Filesystem (2021)

#47

Post by RetroTechie »

frizlab wrote: post #24
RetroTechie wrote: post #21

Useful enough that it should be an OS-level standard feature, imho. Unix-like OSes allow mounting disk images to explore their contents. But there's many more file formats where exploring files-inside-files is useful. Compressed archives, for one. Some file managers support those, but (imho) application-level is not the optimal layer to put this functionality. Could be implemented with a kind of driver-per-filetype.

Honest question: How is this useful? I don’t see any use-case where this would come in handy.

It allows you to use any tool available for regular files, on the files-in-files as well.

As opposed to extract contents and then work on that (requiring extra steps + disk space). Or be limited to what specialized utilities support.

Re: The File Filesystem (2021)

#48

Post by throwway120385 »

codetrotter wrote: post #46
AgentME wrote: post #45

Earlier quoted context omitted.

/tmp isn't stored in memory; it's usually a normal on-disk filesystem that's cleared regularly. You want /dev/shm instead, which is a purely in-memory filesystem on normal Linux systems.

> /tmp isn't stored in memory It is if your system uses tmpfs for /tmp https://en.wikipedia.org/wiki/Tmpfs

The point they were trying to make is that it doesn't have to be, and it isn't in several of the Linux systems I've used over the years. Assuming that it is is a bad idea.

Re: The File Filesystem (2021)

#49

Post by skissane »

amiga386 wrote: post #10
jasonpeacock wrote: post #5

You can also mount your git commits as a filesystem: https://jvns.ca/blog/2023/12/04/mounting-git-commits-as-fold... (previously: https://news.ycombinator.com/item?id=38527866 )

All you need now is a giant pile of rules for which revisions to select and you have the unholy demon that is Rational ClearCase

I always thought Oracle ADE was a cooler demon. Shame the internal talk about productising it never went anywhere.

Re: The File Filesystem (2021)

#50

Post by skissane »

duped wrote: post #41
Groxx wrote: post #29

Earlier quoted context omitted.

Does https://osxfuse.github.io/ cover this? Or is there some fundamental issue? (beyond "it's not built in")

Well that requires a kext so it's a nonstarter, and fuse-t uses NFS which is extremely janky and unreliable on MacOS. The fundamental issue is that macOS doesn't provide an API for this natively.

> fuse-t uses NFS which is extremely janky and unreliable on MacOS

I was wondering what issues you were talking about, and then I found this - https://github.com/macos-fuse-t/fuse-t/issues/45 - data corruption

> The fundamental issue is that macOS doesn't provide an API for this natively.

The API is there, Apple just doesn’t want to give anyone outside of Apple the entitlement that lets them use it. I don’t understand why Apple won’t.

Well, I understand that would require them to document it, ship public headers, and support it for external developers - but why not?

Post reply on HN