Live data from Hacker News

Designing better file organization around tags, not hierarchies (2017)

nayuki.io

41–50 of 169 posts

Re: Designing better file organization around tags, not hierarchies (2017)

#41
post #34
post #25

This is good thinking, and mostly overlaps with what I've tried to do a few times, so I would love to see it finally happen somehow. For example, the Newton storage system I worked on at Apple 1990-1996 was based on separating organization from storage so we could have multiple (tag and/or hierarchy) organization systems. That eventually became the "soup" system in the shipping Newton OS, where objects were retrieved…

I think most people would agree a tag filesystem or a similar concept is a great idea. I have wanted one myself for a long time. Yet, for some reason, it doesn't take off. Do you think it is just a problem of implementation?

Hierarchical file systems allow you to say for sure where a file isn't. Every tagging system I've ever played with has turned out to be a mess in actual use compared to the simplicity a hierarchy provides.

(I am just a user, not a developer of filesystems, so this may be a naive opinion.)

Re: Designing better file organization around tags, not hierarchies (2017)

#42
post #33

Earlier quoted context omitted.

How do you make sure things are tagged properly though? A file must reside within a folder, even if it's a default location, which forces a user to think about the folder where the file is stored. With tags, a user could very easily forget one tag on a file, and now any filtering on that tag is never going to be aware of the new file existing. What if you find another picture somewhere from your trip, but you forget…

Manual tags require a lot of curation and upkeep, but some "tags" are really just restatements of attributes or facts about a file, like search filters, e.g. ("Pictures downloaded from the web on 2018-04-05", "Files created during installation of World of Warcraft", "Files opened in the last two weeks"). In fact, a tag-based document filesystem is largely useless without powerful search, where tag keys and values can…

I suppose I should mention that while MacOS search is far from perfect, Apple put quite a lot of working into this kind of autotagging.

Here's one a screenshot showing a tiny portion of the tags available in search https://imgur.com/a/AYm5V

Re: Designing better file organization around tags, not hierarchies (2017)

#46
post #6

Tagging is the first step, but how do you know if you don't have overlapping or duplicate tags, say country and folk music? If you need something that fits both categories, you eventually start designing taxonomies and eventually ontologies, there's just no end to it. I think tagging is a sensible, lightweight approach, but it has limitations...

I found this to be a real blocker every time I tried to tag something.

However, it’s a complete non-issue if the tags are automatically generated and updated.

I first noticed this in the iOS Photos app, which automatically generates collections such as “Weekend in Paris” and “Pictures of Bob”. The accuracy is good enough to be useful and can be improved automatically in future releases without requiring user interaction.

Re: Designing better file organization around tags, not hierarchies (2017)

#48
post #34
post #25

This is good thinking, and mostly overlaps with what I've tried to do a few times, so I would love to see it finally happen somehow. For example, the Newton storage system I worked on at Apple 1990-1996 was based on separating organization from storage so we could have multiple (tag and/or hierarchy) organization systems. That eventually became the "soup" system in the shipping Newton OS, where objects were retrieved…

I think most people would agree a tag filesystem or a similar concept is a great idea. I have wanted one myself for a long time. Yet, for some reason, it doesn't take off. Do you think it is just a problem of implementation?

A few things I can see off the top of my head that would need to be solved for tag filesystems to be able to take off:

1. Inertia. Most software assumes hierarchical filesystems, and assumes it can control some portion of that hierarchy. This includes things ranging from search paths for various things ($PATH for binaries, library paths, etc), temporary files, preferences files for applications, assumptions made about hierarchical filesystems in archive formats like ZIP and TAR, etc.

2. Permissions. With a hierarchical filesystem, you can apply permissions on higher levels of the hierarchy to control access to lower levels, and you can have various forms of permission inheritance to control permissions on new files. Need a design for how to do that on tag filesystems.

3. Mounting. Filesystems come and go; some are on your OS drive, some are on removable media, some are network filesystems. Hierarchical filesystems means that each one has a single root, and it's easy to tell where the boundaries are.

4. Tagging taxonomy. What kinds of tags do you use? What happens if you mount a filesystem in which someone else used a different tagging taxonomy than you used? Who controls different parts of the tag space? What happens if you import an archive of material which uses a different tagging scheme than you use?

5. Projects. How do you group files of different types with different tags into discrete projects? How do you bundle related files together? How often would you want to see files by arbitrary tag lumped together, rather than looking in particular projects that have a pre-defined structure?

6. UI. How do you browse tags? How do you refine down? In many cases, rather than a general purpose tags based interface, you actually want media-specific browsers, like ones specialized for music which let you browse by artist, album, playlist, etc, or photo galleries that can show you previews of the photos, or video browsers which can show projects, bins, and sequences (for video editing), or IDEs which can either show you file hierarchy or allow you to browse by class, function, etc.

7. And finally, why is a tag-based filesystem necessary for this? What is wrong with the current approach, in which there are special purpose applications which can index, tag, and display certain types of media in certain ways? For instance, you can use your text editor or IDE to navigate among files within development projects, iTunes or Play Music or whatever to browse your music, Lightroom or Darktable or iPhoto or Google Photos to manage your pictures, iMovie or Final Cut or Premiere or Avid for browsing and managing your video, and so on. They all frequently have some way of tagging files, but also have specialized UIs for browsing the specific types of files they are defined for without having to do explicit tagging, and the actual files are just stored on a normal hierarchical filesystem.

There are a couple of good thoughts in the original post, but a lot is handwaved away, such as mutability of files, which is an incredibly important use case, for a huge amount of what files are used for today. Lots of people have brought up the idea of making tag based or database based filesystems, such as the failed WinFS effort (https://en.wikipedia.org/wiki/WinFS), but it's actually a pretty big problem to solve.

Re: Designing better file organization around tags, not hierarchies (2017)

#49
ALL available meta data should be exposed and forced into tags, categories and folders.

Move everything file-like into the file system. Make emails into folders with tagged files. Link Torrents to their files and folders. Treat zips like folders. etc

Bit more on date range sliders, colors, files as tags and 3d models here:

https://steemit.com/filesystem/@gaby-de-wilde/how-a-file-sys...

Re: Designing better file organization around tags, not hierarchies (2017)

#50

>But fundamentally, there is a mismatch between the narrowness of hierarchies and the rich structure of human knowledge, and the proposed system will not presuppose the features of HFSes. This hits the nail on the head ! All the fileSystems I had to work with are fine as engineering tools. By that I mean using them as an engineer works just fine, their own implementation is off topic. As a user though. What the hell…

On OSX these issues have been solved for quite a few years.

1. Press Apple-Space.

2. Type "draft3".

3. Select first item.

And for finding "all my pictures" just create a Smart Folder based on a series of rules e.g. "File Type = JPG".

Post reply on HN