Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Designing better file organization around tags, not hierarchies (2017)

nayuki.io

Post reply on HN
Print view View original

31–40 of 169 posts

Re: Designing better file organization around tags, not hierarchies (2017)

#31

Post by discreteevent »

wrs wrote: post #25

This is good thinking, and mostly overlaps with what I've tried to do a few times, so I would love to see it finally happen somehow. For example, the Newton storage system I worked on at Apple 1990-1996 was based on separating organization from storage so we could have multiple (tag and/or hierarchy) organization systems. That eventually became the "soup" system in the shipping Newton OS, where objects were retrieved…

Would you have time to say a bit more about why WinFS failed from your point of view?

Re: Designing better file organization around tags, not hierarchies (2017)

#33

Post by niftich »

Benjammer wrote: post #15
on_and_off wrote: post #5

>But fundamentally, there is a mismatch between the narrowness of hierarchies and the rich structure of human knowledge, and the proposed system will not presuppose the features of HFSes. This hits the nail on the head ! All the fileSystems I had to work with are fine as engineering tools. By that I mean using them as an engineer works just fine, their own implementation is off topic. As a user though. What the hell…

How do you make sure things are tagged properly though? A file must reside within a folder, even if it's a default location, which forces a user to think about the folder where the file is stored. With tags, a user could very easily forget one tag on a file, and now any filtering on that tag is never going to be aware of the new file existing. What if you find another picture somewhere from your trip, but you forget…

Manual tags require a lot of curation and upkeep, but some "tags" are really just restatements of attributes or facts about a file, like search filters, e.g. ("Pictures downloaded from the web on 2018-04-05", "Files created during installation of World of Warcraft", "Files opened in the last two weeks").

In fact, a tag-based document filesystem is largely useless without powerful search, where tag keys and values can be searched at will.

Re: Designing better file organization around tags, not hierarchies (2017)

#34

Post by yiyus »

wrs wrote: post #25

This is good thinking, and mostly overlaps with what I've tried to do a few times, so I would love to see it finally happen somehow. For example, the Newton storage system I worked on at Apple 1990-1996 was based on separating organization from storage so we could have multiple (tag and/or hierarchy) organization systems. That eventually became the "soup" system in the shipping Newton OS, where objects were retrieved…

I think most people would agree a tag filesystem or a similar concept is a great idea. I have wanted one myself for a long time. Yet, for some reason, it doesn't take off. Do you think it is just a problem of implementation?

Re: Designing better file organization around tags, not hierarchies (2017)

#35

Post by Nadya »

I've used a self-hosted Booru for all of my image sorting. It took me roughly 2 months to upload and tag 53,000 images and another week of cleaning up rarely used or redundant tags. Since I used it outside of the Artist/Series/Character hierarchies my Artist/Series/Character hierarchies refer to Topic/Subject/Details.

For example, visualizations of various algorithms would be filed under:

Topic: Computer Science / Subjects: Algorithms, Visualizations / Data: {Algorithm Name}

By personal restriction - something may only be filed under one topic, no more than three subjects, and can have as much data as is relevant. It gets stored under what I believe to be the primary subject.

The #1 problem is I add files to my filesystem without uploading and tagging them to the Booru. Also, since the only open-source Booru software I could find is quite dated/buggy, I'm often fighting the Booru for how I use it. Now that I think about it, this might be a good problem for me to solve myself.

Re: Designing better file organization around tags, not hierarchies (2017)

#36

Post by mjevans »

megaman22 wrote: post #29
on_and_off wrote: post #5

>But fundamentally, there is a mismatch between the narrowness of hierarchies and the rich structure of human knowledge, and the proposed system will not presuppose the features of HFSes. This hits the nail on the head ! All the fileSystems I had to work with are fine as engineering tools. By that I mean using them as an engineer works just fine, their own implementation is off topic. As a user though. What the hell…

My problem, at least with Windows, is that it's an unholy mishmash of the two ideas. I really just want something like a Unix filesystem, with a single root. Just with things coming off that root that actually make some goddamn sense and aren't relics from the PDP days. Yet on Windows I have different drives that act as independent file trees, and then I have special folders that are handled somewhat differently, app…

It doesn't help that Windows /never/ (still hasn't) developed an actual standard library abstraction of what the filesystem is. In the Unix world your standard library either handles all supported filesystems for you, or more often, just provides a working abstraction of that via the kernel's own VFS abstraction of the supported filesystems.

Literally, in the UNIX/POSIX world there is exactly one way to open a file for a normal application, irrespective of the storage backing for that file and what filesystem might contain it.

(IIRC, it's been a while) On Windows the closest a programmer can get to this is opening up a file via calls specific to NTFS (since it's now the predominant filesystem), FAT (most flash drives), eXFAT (other flash drives), ISO9660(etc)/UDF for optical media, or something else for SMB/CIFS mounts.

For many years I've thought MS should just make a VFS that presents everything as (virtually) NTFS, and use that as the standard library abstraction.

Which of course has very little relation to actually tagging/organizing/search/recall files in a more human way. The biggest hurdle there is probably getting humans to tag/file things correctly and verbosely enough. MOST humans simply can't do a good job at this. For at least 60%, maybe as high as 80 or 90% of the population, it is /literally/ beyond their technical capability to do a good job (at least without a lot of help).

Re: Designing better file organization around tags, not hierarchies (2017)

#37

Post by Angostura »

wrs wrote: post #25

This is good thinking, and mostly overlaps with what I've tried to do a few times, so I would love to see it finally happen somehow. For example, the Newton storage system I worked on at Apple 1990-1996 was based on separating organization from storage so we could have multiple (tag and/or hierarchy) organization systems. That eventually became the "soup" system in the shipping Newton OS, where objects were retrieved…

If you were one of the people behind the ideas in Soup, I salute you.

Re: Designing better file organization around tags, not hierarchies (2017)

#38

Post by kpil »

I think this is fundamentally NOT how humans remembers things - I think we are masters in "geospatial" memory compared to abstract unrelated concepts, and "geospatial" memory is probably organised in hierarchies.

Worse, it's next to impossible to efficiently explore a large tag-cloud compared to a hierarchical structure, which means it's much harder to learn about things organised in a tag-cloud compared to hierarchical tree, or a graph that is mostly tree-like.

As an example; it totally breaks the xkcd techsupport cheat sheet - you end up in the "click one at random" branch basically all the time. https://xkcd.com/627/

Obviously, tagging things is good too, but file systems and computers should emphasise the tree (a better one than the Windows file strucutre though)- rather than inventing a confusing cloud/fog of unrelated things.

Re: Designing better file organization around tags, not hierarchies (2017)

#39

Post by jmiserez »

alanbernstein wrote: post #30
xkcd-sucks wrote: post #22

Earlier quoted context omitted.

> Any sufficiently complicated C or Fortran program contains an ad-hoc, informally-specified, bug-ridden, slow implementation of half of Common Lisp. and any tagging scheme sufficiently good enough to describe the real world will implicitly create a hierarchical ontology

But it will create multiple hierarchies, as opposed to the single hierarchy to which you are constrained by a conventional filesystem.

Do you see that as a positive or a negative?

Re: Designing better file organization around tags, not hierarchies (2017)

#40

Post by fouc »

I'm a big believer in tags. I tried to make a tag-based tool that merely relies on directory names as tags, so ~/t/.tag1/.tag2/tag3/your-file-or-directory and moves your files around so that the tag directories are always organized by tag counts.

I had the idea there would be a series of tmv tcd tls commands that work with the tag directory structure.

https://github.com/foucist/tagmv

Warning - The regexp I'm using is likely broken, I suspect directories with .git/ or other dot-directories in them causes issues. It sometimes causes the .git/ innards to be moved out into the project directory or something like that. I never got around to fixing it.

Post reply on HN