Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Designing better file organization around tags, not hierarchies (2017)

nayuki.io

Post reply on HN
Print view View original

131–140 of 169 posts

Re: Designing better file organization around tags, not hierarchies (2017)

#131

Post by Izkata »

TeMPOraL wrote: post #97
enobrev wrote: post #89

Earlier quoted context omitted.

I absolutely agree with this, except to add that it seems, in theory, in the case of a 100% tag-based file-system, there would never (*very rarely) be a flat list where you have to scroll through millions of files. The UX of a single flat list with millions of files named DCIMxxxx.jpg is a limitation of the current format and doesn't make sense when so much information can be generated about our files upon creation.…

Part of the adoption problem here would be trust. Hierarchical filesystems are something we're used to, and we can trust they're implemented correctly. That means, if I visit a folder and see some files, I know what I see is all of the files there (+/- hidden file settings); if something is missing, it's not there, period. Tag search is a search. Can be broken. Can be optimized in a way that causes it to lie. I look…

> Tag search is a search. Can be broken. Can be optimized in a way that causes it to lie. I look at the results, and I'm not sure if they're complete. Maybe the file I'm looking for is really not there, or maybe the search gave up too early. Or the tag was slightly misformatted?

Google Docs, while not exactly a tag system, wants you to search instead of use a hierarchy, and so is my go-to example: A good 90+% of the time, I can't find something I know is there and have to ask a co-worker for a link.

The missing consideration when it comes to tags is simple discoverability. You have to already know enough about what you're looking for in order to find it. A hierarchical system lets you do systematic browsing.

Re: Designing better file organization around tags, not hierarchies (2017)

#132

Post by enobrev »

TeMPOraL wrote: post #126
enobrev wrote: post #124

Earlier quoted context omitted.

You make excellent points, although I think I the issues you raise exist in our current filesystems as well. Provided the FS is indexed properly, opening a tag should show all files associated with that tag immediately. Just like opening (or listing) a directory does. And in the case of search, it absolutely sucks on today's filesystems. Don't get me wrong, find and grep are incredible tools. I simply mean that it's…

> And in the case of search, it absolutely sucks on today's filesystems. It does. You mention grep, I'd even mention find - half of the time I'm wondering whether it has searched everything I wanted, or I misspelled the command. Or file search in Windows (Vista+) - I just don't trust it; I'm pretty sure it missed some data in the past for one reason or another. Now with traditional file systems, I at least have the f…

> I'd only have search - so it better be trustworthy

Absolutely agreed. It should be as reliable and as immediate as what we have now.

> I at least have the file tree

I don't know how this would work in practice, but I'm imagining something where, UX-wise, a tag-based FS could act very much like what we're already used to. Google was very much on this track in their early versions of "labels" in gmail and google drive (shame they've slowly moved away from it)

Just last night I used some desktop app I found to tag a few thousand scanned documents so I could do my taxes this morning (researching my options is how I ended up finding this article). Once they were all tagged, I was able to traverse in a very familiar way.

At "root", there's too much noise, but as soon as I pick a tag, say "2017" - now I have whittled down my available tags. And then I pick "receipts". Smaller list of files and a smaller list of tags. And then "restaurants". And then "business".

That seems quite a bit like a hierarchy to me. The subset of tags that are related to the first one I chose act just like sub-directories. The UI could work exactly like what we already know and love. As we know it now, I would have ended up at ./2017/receipts/restaurants/business.

Of course with directories, that's the only way I could organize my files. But if we're working with tags, I would get the exact same results going to:

/business/receipts/2017/restaurants/

/receipts/2017/business/restaurants/

You get the idea. But, I could also potentially do something like:

/receipts/2017/client_1+client_2+client_5

or

/2017/receipts/business+!client_3

Now, still within the realm of a directory structure - even using terminal commands we're all familiar with and a bit of extra sugar - I have access to more features. I can't merge directories in a tree. Not that easily, anyway. But in this case I can `cd` into a directory of exactly what I want in a familiar way without trying to remember if what I'm looking for is in ~/Dropbox/receipts/2017 or ~/Documents/business/client_1/receipts.

It's in both. "Dropbox" and "Documents" are no longer necessary. Nor is ~/.

Re: Designing better file organization around tags, not hierarchies (2017)

#133

Post by cesarb »

foobarbecue wrote: post #74
jessmartin wrote: post #54

Have you considered a file system organized as a timeline that _also_ supports tagging? I find one of the key concepts that's not a first-class concept is _when_ the file was modified. Rather than a file-and-folder physical analogy for the file system UI, I think a timeline-oriented UI could present some advantages for the way that humans actually think and work. Tags would be a helpful orthogonal organization scheme…

This might be a good idea if clocks were reliable.

That used to be more of a problem in the past. Nowadays, it's very common for a computer to have its clock synchronized via either NTP or a cell phone network.

Re: Designing better file organization around tags, not hierarchies (2017)

#134

Post by uryga »

jmiserez wrote: post #39
alanbernstein wrote: post #30

Earlier quoted context omitted.

But it will create multiple hierarchies, as opposed to the single hierarchy to which you are constrained by a conventional filesystem.

Do you see that as a positive or a negative?

Not GP, but I'd definitely call that a positive! That's one of my issues with single-inheritance OOP too... There are many hierarchies you might want depending on the aspects you're interested in.

"A glass of juice is certainly a drink, but it is also a source of nutrition -- but not every drink is. There's not much nutrition in a glass of water. Likewise, there are many sources of nutrition which are not drinks. [...] A cup of tea is technically a source of energy (it is hot, it contains thermal energy), but so is a battery. Do they have a common base class?"[1]

(I remember a better example using soft drinks, but I couldn't find it with a quick google)

[1](https://stackoverflow.com/a/1079003/5534735)

Re: Designing better file organization around tags, not hierarchies (2017)

#135

Post by magduf »

reacweb wrote: post #107
nayuki wrote: post #91

Hello everyone, thank you for all the comments. Seeing this on the HN front page caught me by surprise. In the past year I shared this article publicly (Reddit) and privately (with tech-savvy acquaintances) for comment, and the general sentiment I received was that these ideas were not ready to be read by a mass audience. The article is way too long and pulls in many disparate ideas; it explains both why traditional…

Hello, I am trying to make an application for my wife to manage embroideries. I encounter almost all your issues. My wife has thousands of embroideries downloaded from internet. There are many duplicates (filenames not unique because of internationalisation and special characters). She needs to add tags to help search. She also needs groups of tags (tiger belongs to animals, ...). She also has metadata (origin of the…

How large are these embroidery files? I feel like this is something that a SQL database might be able to help with. It's probably not an ideal solution, but remember that a filesystem is really nothing more than a database, which organizes files on-disk in a particular way and indexes them so that you can find them. It obviously has serious limitations due to its hierarchical nature, which is why relational databases were invented, so using existing tools it's probably quite feasible to create an application that uses a SQL database to index all the embroidery files and store all this data on them (license, whether it's a derivative of another one, and tags), and then if they're too large to just store in the DB itself, just point to files in the regular filesystem.

Re: Designing better file organization around tags, not hierarchies (2017)

#136

Post by jonnycomputer »

jonnycomputer wrote: post #70
abathur wrote: post #52

I've wanted something along these lines for a long time as well. I have trouble drawing hard lines and distinctions (this is pervasive; things like having a "favorite" anything, or the desire to debate what genres a song or movie fall into, are rather alien to me). This makes picking "one" place for something difficult. Because these are fine/fuzzy distinctions for me, it's also tricky to reason my way back to where…

I don't see why one couldn't have a tag system whose scope is defined by its location in a hierarchical structure.

apparently someone thought that the notion of a tag system with a semantic scope bound to a file tree hierarchy is such a ridiculous--nay, offensive!--idea that they had to downvote me for it, but couldn't bother to address it on the merits. The problem I see with tag based structures is that tags are global in scope, and so, tags have to mean one thing and one thing only; on the other hand, having one set of tags for a photo directory subtree, and another for my code repository makes a hell of a lot of sense to me.

Re: Designing better file organization around tags, not hierarchies (2017)

#137

Post by dgreensp »

egypturnash wrote: post #92

Every time I read an article about attempts at non-hierarchical filesystems, I try to figure out how I'd take the huge piles of stuff I generate when I'm drawing (and publishing) a graphic novel and reorganize it under tags. It's never pretty. Like, okay, sure, I tag everything with the name of the project, that's a no-brainer. But if I just do that then I get the hundreds of files I generate (one per page) mixed up…

I can strain my imagination and optimistically say that with a suitable UI, the tag-based file browser would be no worse than the hierarchical one — using tags to categorize your files in a not-too-coarse, not-too-fine way, and maybe even having more flexibility in how you organize and browse your files. Do you ever put information in the file name that could be a tag, or make up meaningless file names when there are few enough files related to a particular page, for example? Or put information in names that could be attached as some sort of comment or notes metadata instead? But I think you bring up some important issues when it comes to organizing your stuff.

Would it be better if, instead of having kitchen drawers and cabinets, we had a tag-based system, because, you know, the structure of human knowledge and all that? Why should I be forced to put a utensil in zero or one drawers? Actually, the cabinets and drawers system is nice because you have a sense of “place” — you can think intuitively about where an item is, where an item goes; even if where an item goes is a somewhat arbitrary choice, at least you know exactly what decisions have to be made (which drawer and which compartment in the drawer organizer, for example) to put it away. You can also do a traversal through the cabinets and drawers to see what objects are being stored and how they are being organized, and any time you open a cabinet, you are focusing on a different set of objects. Imagine if you had 10 cabinets and 10 objects in each cabinet, but you actually only have 20 objects total. Every time you open a different cabinet, you see a different 10 of those 20 items. Confusing.

I wonder if it would help to have required tags, and exclusive tags. Files with tag X must have tags of type A, B, and C, and may have other nonessential tags. X would be something like, “is a project file for some project,” and A could be the type of project name tags, and so on.

Re: Designing better file organization around tags, not hierarchies (2017)

#138

Post by dgreensp »

I found this to be a great collection of insights!

My criticisms:

Organizing your files and digital “stuff” has very little to do with the “rich structure of human knowledge,” to me, any more than organizing your kitchen or garage is an exercise in philosophy. The goal should be as usable a system as possible, full stop. Now, the actual content of the article is extremely practically oriented, so I have no beef with that. I just think people get carried away with the idea that storing a file is “representing knowledge,” and it takes them in weird directions like trying to create elaborate universal ontologies. The question is, is it easier or harder to find your files, and save your files?

Whenever a phrase like “representing knowledge” or “augmenting intelligence” comes up, it’s like everyone gets a boner, and then moves on to something unrelated, like (hopefully) usability.

Mutability: Everything changes. The only way to have immutable facts is to have timestamps. Image hosting sites, message boards, etc are misleading examples of file storage because they are really means of publishing. When you publish something, and people link to it, there’s a case for thinking of immutability as the default, though even then, most things that can be published can be retracted or edited. This comment can be edited after I publish it. I think true immutability as a default, for files as opposed to time-stamped facts, only makes sense in a very narrow domain.

Re: Designing better file organization around tags, not hierarchies (2017)

#139

Post by 411mrc »

sharpercoder wrote: post #112
411mrc wrote: post #96

Earlier quoted context omitted.

Thanks. My subtle point was why use tags when you could use search? Search engines index the whole internet - millions of hierarchical file systems of all kinds. Why reinvent file systems? I don't use Mac. Windows 10 "Cortana" is no match for where Google Desktop was 10 years ago. I rarely use it unless I have to. Google searched the contents. Cortana just searches filenames and tries to route searches to the web. Li…

Google Desktop was indeed a great solution partially solving the tag problem in Windows. A good tagging system trumps search as you can make the results much better determinate. The ability to create your own namespace and then organize every file into that namespace ensures encapsulation. In a search result, you almost always have to filter out irrelevant items. Good working search solves the problem pragmatically t…

I am far too lazy to tag every file and I would despise such a system if forced into it. Some files I want to keep realizing I may never need them again. Tagging is a time waste. It's also difficult to anticipate future use and what tags are helpful.

Now a tagging system that could be built over time from search results could be very useful. Apply tags as you go in batches in other words to aid future searches.

If I were designing a new OS, I'd force each piece of software to auto-tag and index the file contents in meta tags and feed it to a global OS search function.

Re: Designing better file organization around tags, not hierarchies (2017)

#140

Post by kqr »

reacweb wrote: post #107
nayuki wrote: post #91

Hello everyone, thank you for all the comments. Seeing this on the HN front page caught me by surprise. In the past year I shared this article publicly (Reddit) and privately (with tech-savvy acquaintances) for comment, and the general sentiment I received was that these ideas were not ready to be read by a mass audience. The article is way too long and pulls in many disparate ideas; it explains both why traditional…

Hello, I am trying to make an application for my wife to manage embroideries. I encounter almost all your issues. My wife has thousands of embroideries downloaded from internet. There are many duplicates (filenames not unique because of internationalisation and special characters). She needs to add tags to help search. She also needs groups of tags (tiger belongs to animals, ...). She also has metadata (origin of the…

To be pragmatic, it sounds like she needs a relational database more than a filesystem. (Whether or not filesystems should be more like relational databases is a hypothetical at present.)

If she actually manages that sort of data outside of an RDBMS I do suffer with her.

Post reply on HN