Taxonomy Is Hard
autodidacts.io
Taxonomy Is Hard
1–10 of 83 posts
Re: Taxonomy Is Hard
#2Re: Taxonomy Is Hard
#3Well, if symlinks seem inelegant, there are hard links as well, you know. Ultimate tag system: for each tag, make a directory with corresponding name, fill with hard links to appropriate files stored in one big directory of mud.
Of course, it all have to fit into a single disk drive, and actually deleting a file is difficult but hey, that must be easily solvable, details are left as an exercise for the reader.
Re: Taxonomy Is Hard
#4Taxonomy is a demon which separates people into perfectionists and non-perfectionists just before dragging both the kinds to hell of exceptions, weird relations and impractical location. Perfectionists get stuck spending infinite amounts of time engineering the taxonomy, non-perfections face the quirks later.
Tags are better but can turn out to be even harder (for similar reasons, amplified combinatorially).
Labels are the most practical. The GMail inventor was genius. Folders must begone (except for system files).
Re: Taxonomy Is Hard
#5Tagging and categorizing are two subtly different things to do. Having dealt with a lot of real world data, all I can say is that getting your hands on consistently tagged or categorized data is hard and gets harder the more data sources you have.
A real world example of how tagging can be both super useful and get out of hand is open street maps. The only meta data allowed in there are tags. The OSM community depends on people using tags correctly. But of course the tagging is incomplete, inconsistent, and subject to regional and data source specific variations. Which makes interpreting the tags a bit of a dark art.
But it still adds up to a very complete and rapidly evolving world map. So, there's that.
I've had the pleasure of seeing the documentation for Navteq's internal meta datas schema that they used for their maps while I still worked for Nokia Maps, which at the time owned Navteq. These days the whole thing is known as Here Maps. This was a PDF of around 4K pages. Thousands of attributes. Lots of weird little details related to traffic lights, subway entrances and exits, and other features you have on maps. This stuff gets complicated quickly.
Two very different approaches to the same problem. I think I like the OSM way a bit better. Neither is easy.
My job at the time was trying to align some of that data with data we got from external data sources like TripAdvisor, Qype, HRS, and a few others. Way too much time got invested in dumbing down categories, mapping one to the other, and trying to make sense of stuff. We had lots of issues with duplicate POIs because of all sorts of subtle differences in how different data sources were annotated with categories, tags, etc. Some of the data just isn't that good, complete, or consistent and you have to deal with that.
Re: Taxonomy Is Hard
#6All true. Taxonomy is indeed hard. But, does it actually matter?
It seems what matters is not how files are stored/organized, but how one can find the files one is looking for. Taxonomy is mostly a search problem.
Yet, although one is usually capable of remembering specific or unique details about a file, it's still incredibly hard to search for a file or its contents effectively.
Dropbox, to pick just one example, ditched all its advanced search functionalities a couple of years ago and only allows to search for any keyword (not all of them), with forced stemming, and no filtering of the results.
I seem to remember there were startups a decade ago trying to address the search problem on the desktop but they all got acquired or folded; I don't understand why. The problem is real and seems quite solvable; yet it seems there's no actual market for it. It's a bit of a mystery.
Re: Taxonomy Is Hard
#7Plain old unix find with grep, locate or xapian help me navigate.
Remembering to run updatedb is the pain. I won't put it in cron
because grinding disks every night annoys the hell out of me.
Re: Taxonomy Is Hard
#8Categorization is order, tagging is chaos. No order is perfect, but it enables people to talk about the same entities and to agree on certain ideas. Tagging is when everyone invents their own labels and conventions, requiring tons of "smart" algorithms to make sense of the mess.
Re: Taxonomy Is Hard
#9All true. Taxonomy is indeed hard. But, does it actually matter? It seems what matters is not how files are stored/organized, but how one can find the files one is looking for. Taxonomy is mostly a search problem. Yet, although one is usually capable of remembering specific or unique details about a file, it's still incredibly hard to search for a file or its contents effectively. Dropbox, to pick just one example, d…
> startups a decade ago trying to address the search problem on the desktop but they all got acquired or folded; I don't understand why.
Work migrated to the cloud.
Re: Taxonomy Is Hard
#10I store my files chronologically. Every time I want to store something I want to keep, I make a folder with the name format "2022-11-01 something something". The text here is just a short description in natural language. If I feel like it I add a tag here like invoice or photos. The point is to make it searchable. I easily find most things I am looking for with Directory Opus and Recoll.