Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Designing better file organization around tags, not hierarchies (2017)

nayuki.io

Post reply on HN
Print view View original

121–130 of 169 posts

Re: Designing better file organization around tags, not hierarchies (2017)

#121

Post by pbalau »

niftich wrote: post #33
Benjammer wrote: post #15

Earlier quoted context omitted.

How do you make sure things are tagged properly though? A file must reside within a folder, even if it's a default location, which forces a user to think about the folder where the file is stored. With tags, a user could very easily forget one tag on a file, and now any filtering on that tag is never going to be aware of the new file existing. What if you find another picture somewhere from your trip, but you forget…

Manual tags require a lot of curation and upkeep, but some "tags" are really just restatements of attributes or facts about a file, like search filters, e.g. ("Pictures downloaded from the web on 2018-04-05", "Files created during installation of World of Warcraft", "Files opened in the last two weeks"). In fact, a tag-based document filesystem is largely useless without powerful search, where tag keys and values can…

Maybe not use the same file system for everything?

Re: Designing better file organization around tags, not hierarchies (2017)

#123

Post by projektfu »

mjevans wrote: post #64
TylerE wrote: post #56

Earlier quoted context omitted.

What do you mean? fopen exists on windows.

It seems you are correct. I /do/ remember something a /long/ time ago having this issue, but even some quick searching reveals that MSDN documents fopen existing back in 2008, and the notes supporting UNCs. I wonder how the underlying support is handled.

CreateFile can open anything on any mounted filesystem, as well as things like raw drives and named pipes.

Re: Designing better file organization around tags, not hierarchies (2017)

#124

Post by enobrev »

TeMPOraL wrote: post #97
enobrev wrote: post #89

Earlier quoted context omitted.

I absolutely agree with this, except to add that it seems, in theory, in the case of a 100% tag-based file-system, there would never (*very rarely) be a flat list where you have to scroll through millions of files. The UX of a single flat list with millions of files named DCIMxxxx.jpg is a limitation of the current format and doesn't make sense when so much information can be generated about our files upon creation.…

Part of the adoption problem here would be trust. Hierarchical filesystems are something we're used to, and we can trust they're implemented correctly. That means, if I visit a folder and see some files, I know what I see is all of the files there (+/- hidden file settings); if something is missing, it's not there, period. Tag search is a search. Can be broken. Can be optimized in a way that causes it to lie. I look…

You make excellent points, although I think I the issues you raise exist in our current filesystems as well. Provided the FS is indexed properly, opening a tag should show all files associated with that tag immediately. Just like opening (or listing) a directory does.

And in the case of search, it absolutely sucks on today's filesystems. Don't get me wrong, find and grep are incredible tools. I simply mean that it's not like searching for files beyond the hierarchy is known for the pleasurable UX. The only way I know a grep of a whole drive or deep directory is done is because I get my blinking cursor back.

At the very least with a proper tagging system, we would be inherently familiar with the indexes available to us.

Re: Designing better file organization around tags, not hierarchies (2017)

#125

Post by OscarCunningham »

Benjammer wrote: post #15
on_and_off wrote: post #5

>But fundamentally, there is a mismatch between the narrowness of hierarchies and the rich structure of human knowledge, and the proposed system will not presuppose the features of HFSes. This hits the nail on the head ! All the fileSystems I had to work with are fine as engineering tools. By that I mean using them as an engineer works just fine, their own implementation is off topic. As a user though. What the hell…

How do you make sure things are tagged properly though? A file must reside within a folder, even if it's a default location, which forces a user to think about the folder where the file is stored. With tags, a user could very easily forget one tag on a file, and now any filtering on that tag is never going to be aware of the new file existing. What if you find another picture somewhere from your trip, but you forget…

I imagine the process as being similar to how you would currently add a file.

Now:

Your friend messages you "Here's another pic from Paris xxx". You click on the picture, "Save as". Your file system comes up, you navigate to benjammer/travel/paris/pics. You can see the rest of the pictures from Paris in the directory. You hit save.

With tags:

Your friend messages you "Eiffel Tower :-)". You click on the picture, "Save as". A list of your tags comes up for you to select some. On the other side of the screen are the files that match the selected tags, so that you can see what company your file is going to end up in. They're shown in the order of how many files they tag (that also have the tags you've already selected). The "benjammer" tag is selected by default, as is "picture" and "png" (because your application knows it's been given a png). The "Paris" tag isn't at the top of the list so you type it in and select it. Now "Travel" is at the top of your list of tags, so you add it, along with "June 2017" and "Europe".

Re: Designing better file organization around tags, not hierarchies (2017)

#126

Post by TeMPOraL »

enobrev wrote: post #124
TeMPOraL wrote: post #97

Earlier quoted context omitted.

Part of the adoption problem here would be trust. Hierarchical filesystems are something we're used to, and we can trust they're implemented correctly. That means, if I visit a folder and see some files, I know what I see is all of the files there (+/- hidden file settings); if something is missing, it's not there, period. Tag search is a search. Can be broken. Can be optimized in a way that causes it to lie. I look…

You make excellent points, although I think I the issues you raise exist in our current filesystems as well. Provided the FS is indexed properly, opening a tag should show all files associated with that tag immediately. Just like opening (or listing) a directory does. And in the case of search, it absolutely sucks on today's filesystems. Don't get me wrong, find and grep are incredible tools. I simply mean that it's…

> And in the case of search, it absolutely sucks on today's filesystems.

It does. You mention grep, I'd even mention find - half of the time I'm wondering whether it has searched everything I wanted, or I misspelled the command. Or file search in Windows (Vista+) - I just don't trust it; I'm pretty sure it missed some data in the past for one reason or another.

Now with traditional file systems, I at least have the file tree. With tag-based systems, I'd only have search - so it better be trustworthy, both in reality and UX-wise. It needs to project the feeling of correctness and completeness or results.

> The only way I know a grep of a whole drive or deep directory is done is because I get my blinking cursor back.

The only way I know a find of a whole drive does what it's supposed to be doing is because it emits a stream of "find: `/some/path': Permission denied" messages.

> At the very least with a proper tagging system, we would be inherently familiar with the indexes available to us.

Fair enough.

Re: Designing better file organization around tags, not hierarchies (2017)

#127

Post by tinix »

friendlydude12 wrote: post #106
nayuki wrote: post #91

Hello everyone, thank you for all the comments. Seeing this on the HN front page caught me by surprise. In the past year I shared this article publicly (Reddit) and privately (with tech-savvy acquaintances) for comment, and the general sentiment I received was that these ideas were not ready to be read by a mass audience. The article is way too long and pulls in many disparate ideas; it explains both why traditional…

I think a big problem with metadata-aware “file systems” is that the metadata is lost once the file is exported out of the system. This is a problem with ID3 tags for instance. Another problem is where you make the compromise in the no-mans land between fully fledged data structure and file system. As soon as you start adding meaningful metadata to the file system, it quickly becomes apparent that you want the files…

> the metadata is lost once the file is exported out of the system

As someone who has implemented such a "file system" (or three) for various types of enterprise clients, some of which in turn serve it to other b2b clients of their own... I just have to say... this isn't necessarily a bad thing, and also, it's not necessarily true either. For starters, one can easily give every file a unique uuid, and map that uuid to a spreadsheet full of metadata. Additionally, a little vendor lock in to keep "special features" like management of a bespoke file system isn't necessarily a bad thing, either, if it's in your best interest to keep paying customers. Application specific use cases? Sure... but what isn't? You can build a generic abstract non-hierarchical file system though... easily.

Re: Designing better file organization around tags, not hierarchies (2017)

#128

Post by ausjke »

chadcmulligan wrote: post #119
Retra wrote: post #69

Earlier quoted context omitted.

What would you do with the names?

make an enemies of ausjke list probably

nope, just ask why? you're what you think I guess, I have not made any lists in my whole life and surprised you immediately guessed that.

Re: Designing better file organization around tags, not hierarchies (2017)

#130

Post by sullyj3 »

enobrev wrote: post #89
kijin wrote: post #67

Earlier quoted context omitted.

Tags are fun when you have a few thousand items to test your MVP with. It gets much less fun when you have millions of items with thousands of tags, all on a flat hierarchy. On the other hand, when you're stuck with a flat hierarchy anyway (e.g. thousands of pictures, all named DCIMxxxx.jpg), tags can be more useful. But only if they're automatic. I want the best of both worlds. I want to organize my stuff into folde…

I absolutely agree with this, except to add that it seems, in theory, in the case of a 100% tag-based file-system, there would never (*very rarely) be a flat list where you have to scroll through millions of files. The UX of a single flat list with millions of files named DCIMxxxx.jpg is a limitation of the current format and doesn't make sense when so much information can be generated about our files upon creation.…

I think the parent wasn't concerned about scrolling through large numbers of files, so much as the performance issues associated with querying them.
Post reply on HN