Designing better file organization around tags, not hierarchies (2017)

As I recently inherited a huge, well-tagged music collection (tens of terabytes of files) I am very interested in this. Is there something like it, also supporting .cue files and also storing the original filenames and structure? A mediaplayer agnostic way to access this treasure trove would be the best.

Re: Designing better file organization around tags, not hierarchies (2017)

#5

Post by on_and_off » Thu, Apr 05, 2018, 8:54 PM UTC

>But fundamentally, there is a mismatch between the narrowness of hierarchies and the rich structure of human knowledge, and the proposed system will not presuppose the features of HFSes.

This hits the nail on the head !

All the fileSystems I had to work with are fine as engineering tools. By that I mean using them as an engineer works just fine, their own implementation is off topic.

As a user though.

What the hell !

I don't want to go to c:/users/me/documents/talks/stockholm2018/draft3

I just want to open my document !

I really hope that someday we expose a document based filesystem to the user.

The underlying implementation does not matter, we can always add a layer on top of the hierarchical file system.

I just want to be able to display :

-all the games installed on my system .

-all the pictures

-all of my text documents .

-all of my pictures of Paris

etc

Re: Designing better file organization around tags, not hierarchies (2017)

#6

Post by xchaotic » Thu, Apr 05, 2018, 8:56 PM UTC

Tagging is the first step, but how do you know if you don't have overlapping or duplicate tags, say country and folk music? If you need something that fits both categories, you eventually start designing taxonomies and eventually ontologies, there's just no end to it. I think tagging is a sensible, lightweight approach, but it has limitations...

Re: Designing better file organization around tags, not hierarchies (2017)

#7

Post by Slansitartop » Thu, Apr 05, 2018, 8:57 PM UTC

I never personally used it, but I've heard the BeFS was designed to have significant non-hierarchical use cases:

https://en.wikipedia.org/wiki/Be_File_System:

> [BeFS] includes support for extended file attributes (metadata), with indexing and querying characteristics to provide functionality similar to that of a relational database.

IIRC, this was pretty hyped at the time, but they had to back away from it. I don't know if it was because if the concept was too unfamiliar to people familiar with the hierarchical paradigm or if it didn't work as well in practice as it was imagined.

There's also a book about it written by its designer and now freely available: http://www.nobius.org/dbg/practical-file-system-design.pdf

Re: Designing better file organization around tags, not hierarchies (2017)

#8

Post by roywiggins » Thu, Apr 05, 2018, 9:05 PM UTC

on_and_off wrote: post #5

>But fundamentally, there is a mismatch between the narrowness of hierarchies and the rich structure of human knowledge, and the proposed system will not presuppose the features of HFSes. This hits the nail on the head ! All the fileSystems I had to work with are fine as engineering tools. By that I mean using them as an engineer works just fine, their own implementation is off topic. As a user though. What the hell…

I eventually had to use a series of command-line tools to shepherd my badly-organized photo collection into something like a usable state, and it basically involved finding all the JPEGs on my system, de-duplicating them, and dropping them into folders based on their date-taken metadata.

It was a huge pain, and entirely an artifact of the tyranny of the folder-based filesystem.

Re: Designing better file organization around tags, not hierarchies (2017)

#9

Post by sdhgaiojfsa » Thu, Apr 05, 2018, 9:05 PM UTC

Every time I use a tagging-based system, I become more convinced that tags are what I want for almost all things, not just files.

Re: Designing better file organization around tags, not hierarchies (2017)

#10

Post by fao_ » Thu, Apr 05, 2018, 9:07 PM UTC

I spent last year ruminating on this (Independently. However I find it interesting the article was published around the same time that I was voicing my ideas to a friend on this!) and toying with a few prototypes. This year I committed myself to hacking on a proper implementation (named 'libkoios' and 'koios') of it, using the Extended Filesystem Attributes. What I found interesting is that while there was a lot of prior work systems existing for tagging, none of them use the extended attributes system, which to me feels like a waste. However there are problems with ext(2,3,4)'s implementation of file tags that make it difficult to store a lot of data without compression (I'm storing one bit per tag, which allows fast masking and comparison operations per file), so I guess that is understandable.

I believe that for image-based systems there is 8ch's /hydrus/ (probably the only good thing to come out of the chan-networks). One upshot of there being existing network sharing systems for tags is that it should be possible to scrape them when autotagging things (Nobody. NOBODY, wants to manually tag hundreds of photo memes, which is the main forseeable problem with file tagging).