131–140 of 150 posts
Earlier quoted context omitted.
What problem do you think a DB-as-filesystem solves? The only obvious one that makes any sense at all is cross-file transactions.
The filesystem is a bad database Directories are a shitty underpowered way to organize data? No good transactions Conflation of different needs such as atomic replace vs log-structured I would like to use a better database instead.
Earlier quoted context omitted.
Except that now when you do need a tool like XSLT/XPath, jq, or yq, now you need bash. I use bash lots, but still I'd rather use a better language, like the ones you listed. I'm being slightly hypocritical because I've made plenty of use of the filesystem as a configuration store. In code it's quite easy to stat one path relative to a directory, or open it and read it, so it's very tempting.
You don’t need bash to traverse a file system, are you saying something else?
Earlier quoted context omitted.
That is literally what xattrs are for.
Yes, but they seem fairly limited in terms of userspace programs. How would you use xattrs to produce a filesystem hierarchy that say, listed the same file in multiple folders according to the attributes?
Earlier quoted context omitted.
The filesystem is a bad database Directories are a shitty underpowered way to organize data? No good transactions Conflation of different needs such as atomic replace vs log-structured I would like to use a better database instead.
Would you pay 50x performance decrease?
Earlier quoted context omitted.
Every single time this has been tried it has gone wrong, but sure. Almost all of the operations done on actual filesystems are not database like, they are close to the underlying hardware for practical reasons. If you want a database view, add one in an upper layer.
I don't see how file systems aren't some sort of DBMS, definitely not relational but that wasn't a stated requirement.
Earlier quoted context omitted.
Yes, but they seem fairly limited in terms of userspace programs. How would you use xattrs to produce a filesystem hierarchy that say, listed the same file in multiple folders according to the attributes?
The xattrs are for storing the tag metadata, you’d use other tools (easily composed from shell utilities) to find files that match tags. If you really want it to be in multiple locations, you could make a fuse interface that shows directories full of files matching specific tags.
Yeah, that's the kind of thing that I've wanted, but not really had the programming skill/experience/patience to make. There have been a couple of similar projects, but nothing that seems popular enough to be worthwhile spending time using.
I've been wondering for about 20 years why File Systems basically died and stopped innovating. For example we have lots of hierarchical data structures in the world, and no one seems to have figured out how to let a folder be the storage, instead of always just databases. For example, if we simply had the ability to have "ordered" files inside folders, that would instantly make it practical for a folder structure to…
I agree! I'm sort of exploring "programmable filesystem" concept (using FuseFS) (for some notes, see [1]). Re: ordered files: depends on FS. e.g. filesystems which use B+ trees will tend to have files (in directories) in lexical order. So in some cases you may not need a new FS: echo 'for f in *.txt; do cat "$f"; done' > doc.sh; chmod +x doc.sh => `doc.sh` in dir produces 'documents' (add newlines / breaks as needed,…
Earlier quoted context omitted.
> "Likely the idea that filesystems should run as userspace / unprivileged (or at least limited privilege) processes which would make them, ultimately, indistinguishable from a form of database engine." "Userspace vs not" is a different argument from "consistency vs not" or "atomicity vs not" or "POSIX vs not". Someone still needs to solve that problem. Sure instead of SQLite over POSIX you could implement POSIX over…
> "Userspace vs not" is a different argument from "consistency vs not" or "atomicity vs not" or "POSIX vs not". Someone still needs to solve that problem. Sure instead of SQLite over POSIX you could implement POSIX over SQLite over raw blocks. But you haven't gained anything meaningful. This was an attempt to possibly explain the microkernel point GP made, which only really matters below the FS. > I think this is red…
You seem unhappy with POSIX because its guarantees feel incomplete and ad hoc (they are). You like databases because their guarantees are more robust (also true). DBMS over POSIX enables all the guarantees that you like. I'd want to invoke the end-to-end systems argument here and say that this is how systems are supposed to work: POSIX is closer to the hardware, and as a result it is messier. It's the same reason TCP in-order guarantees are layered above the IP layer.
Some of your points re: how the lower layers work seem incorrect, but that doesn't matter in the interest of the big picture. The suggestion (re: microkernels) seems to be that POSIX has a privileged position in the system stack, and that somehow prevents a stronger alternative from existing. I'd say that your gripes with POSIX may be perfectly justified, but nothing prevents a DBMS from owning a complete block device, completely circumventing the filesystem. POSIX is the default, but it is not really privileged by any means.
Earlier quoted context omitted.
The xattrs are for storing the tag metadata, you’d use other tools (easily composed from shell utilities) to find files that match tags. If you really want it to be in multiple locations, you could make a fuse interface that shows directories full of files matching specific tags.
> If you really want it to be in multiple locations, you could make a fuse interface that shows directories full of files matching specific tags Yeah, that's the kind of thing that I've wanted, but not really had the programming skill/experience/patience to make. There have been a couple of similar projects, but nothing that seems popular enough to be worthwhile spending time using.
If I understand correctly, this is attaching metadata to files in a format that LLMs (or any tool that can understand the semantic embedding vector) can leverage to understand what a file is without having to actually read the contents of the file. That obviously has a lot of interesting use cases, but my first assumption was that this could be used to quickly/easily search your filesystem with some prompt like "Play…
if you go look up how xattrs work, you will understand it's no different than just reading a chunk of the file in question, and in fact can be slower. xattrs are better be forgotten already. it was just as dumb idea as macos resource forks/