Live data from Hacker News

Taxonomy Is Hard

autodidacts.io

61–70 of 83 posts

Re: Taxonomy Is Hard

#61
Categorising information into taxonomies is like trying to hammer a square peg into a round hole; sometimes a necessary but undecidable problem. As someone once said: a book (article, webpage, whatever) is rarely about one thing.

This is a topic that is at the top of my mind, as I grapple to organise my growing gemini/gopher site. Is it better to index, list a table of contents, search, or try to classify it with the (DDC) Dewey Decimal Classification.

The DDC. It has come under criticism, and librarians have justified a lot of their efforts in moving away from it. I doubt that the effort was justified. It boils down to this: you have to put a book in a library somewhere. And that somewhere has to boil down to a taxonomy.

To illustrate the problem, is a book about programming microcontrollers a book about programming, or is it primarily about microcontrollers? The Arduino Cookbook is in DDC 621.3810285536 (yes, really. That's obviously extreme, though). That's part of the electronics section, which seems fair enough to me. So far so good, But "Beginning MicroPython with the Raspberry Pi Pico: Build Electronics ..." is in section 005.13, which is programming. A completely different place. "Programming with STM32: Getting started with the nucleo board" is in 005.262, which is also programming. But why 005.262 rather than 005.13? It almost seems that whoever is classifying these books has no idea what they're doing ;)

I could go on at length about the confusions I have in trying to place my content. In the end, you have to make a somewhat arbitrary decision and just go with it.

Tables of contents work reasonably well within a book. Subjects are often non-intersecting, so they can be treated separately. For the most part, anyway.

A solution which is fairly reasonable is to index your site. Indices are useful because they allow you to take multiple views on something, thereby eliminating the taxonomy problem.

I'm not a great fan of tagging. It is too much of a scattergun approach to my liking. Perhaps some merit, though.

Then there's textual searching. In fact, that's how I relocated some of my notes. So, text search it is, then? Well, not quite. It seemed like a good system for my site which is focussed. It has problems scaling. I don't want millions of results, a la Google, I want a few relevant ones.

This is even a problem with search engines for the gemini and gopher protocols, where nobody is even trying to game the system. You often end up with a lot of similar stuff at the top which I am not interested in.

Oddly, for gemini, I prefer the "Collaborative Directory of Geminispace" over at gemini://cdg.thegonz.net/ , which is a taxonomy of categories, the very thing that I has doubts about.

So, in summary, it's not easy.

Re: Taxonomy Is Hard

#62
post #57

Earlier quoted context omitted.

My thinking is that categorization is a dead horse. Here is a problem: take a car and a truck. Now slowly, bit by bit, morph the car into the truck. At what point is the car a truck? Or is it ever a truck? I, personally, personally cannot see this as a problem that can be solved. Maybe you are better. My conclusion is that composition is a better method. Has a. And composition by capability is even better. Not great,…

> I, personally, personally cannot see this as a problem that can be solved. Maybe you are better. Both library science and cognitive linguistics have solutions for/discuss this problem.

I would say what they call 'composition by capability' is a theory of categorization, in fact similar to Wittgenstein's family resemblance theory.

That said, it's not obvious that the underlying data model needs or should map to a theoretical model for how human cognition works. Maybe it should and it's worth considering the theoretical landscape before setting out, but there are other features that come into play.

I know very little about library science, but I assume they take a more practical (for this task) approach to this that's worth looking into.

Re: Taxonomy Is Hard

#63

Tagging and categorizing are two subtly different things to do. Having dealt with a lot of real world data, all I can say is that getting your hands on consistently tagged or categorized data is hard and gets harder the more data sources you have. A real world example of how tagging can be both super useful and get out of hand is open street maps. The only meta data allowed in there are tags. The OSM community depend…

My thinking is that categorization is a dead horse. Here is a problem: take a car and a truck. Now slowly, bit by bit, morph the car into the truck. At what point is the car a truck? Or is it ever a truck? I, personally, personally cannot see this as a problem that can be solved. Maybe you are better. My conclusion is that composition is a better method. Has a. And composition by capability is even better. Not great,…

> At what point is the car a truck?

when you can use it as a truck (ie. when it can do what the car isn't adequate for).

Re: Taxonomy Is Hard

#64
I agree. Set theory is more powerful and flexible than taxonomic trees (although not perfect). It's why I believe the future is Table Oriented Programming (TOP), where code blocks are either in or managed by RDBMS. Code-centric tools rely too much on file trees and other trees. If you instead try to design your stack and/or language around sets, you'll probably end up with something similar to TOP.

https://news.ycombinator.com/item?id=33413124&p=2#33415249

Re: Taxonomy Is Hard

#65

Tagging and categorizing are two subtly different things to do. Having dealt with a lot of real world data, all I can say is that getting your hands on consistently tagged or categorized data is hard and gets harder the more data sources you have. A real world example of how tagging can be both super useful and get out of hand is open street maps. The only meta data allowed in there are tags. The OSM community depend…

My thinking is that categorization is a dead horse. Here is a problem: take a car and a truck. Now slowly, bit by bit, morph the car into the truck. At what point is the car a truck? Or is it ever a truck? I, personally, personally cannot see this as a problem that can be solved. Maybe you are better. My conclusion is that composition is a better method. Has a. And composition by capability is even better. Not great,…

Discrete categorization of continuous phenomena is always going to be inherently arbitrary. It's important to keep in mind what your goals are. This sort of categorization shouldn't be done with the goal of finding some sort of fundamental platonic truth; that's not going to work. Rather, discrete categorization is performed simply because humans find these categories useful when communicating.

The fuzzy line between truck and car isn't a problem when you approach categorization with this mindset. If you see a bank robber fleeing in an El Camino, you can tell the cops they fled in a "truckish car" or a "carish truck"; you don't have to neatly categorize an El Camino as one or the other to get your point across, but the arbitrary categories still help you communicate the idea.

If you're tagging pictures and come across an El Camino, and you don't have a "coupe utility vehicle" tag, you can simply tag the El Camino as both a truck and a car.

Re: Taxonomy Is Hard

#66
post #36

Tagging, seems like a solution. But isn't. Specific problems with tagging: - Need to tag every file (whereas in folders, you just navigate to the folder and everything you store there is in that folder) - Takes too long - Too much thinking overhead (at the time of storing) - To be effective have to enter the name for all tag entries (e.g. project, type, etc.). If anything is missed for a file, that file will never be…

Maybe automatic tagging with the help of some kind of AI would work?

Doesn't even need to be AI. I've done something very similar to this with text mining, regex, and some xquery/shell. The biggest problem, though, is convincing people that taxonomy doesn't belong in filenames, but that's built in to specifications. Kind of a brick wall, that.

Re: Taxonomy Is Hard

#67
Fun fact: Taxonomist is actually a role at many of the top tech companies. Much of the faceted search experiences are manually determined by taxonomists. Example: search cars allows you to filter by brand, color, engine type, etc vs searching furniture allows filtering by dimensions. Facebook, Walmart, etc. employ a few of these folks.

Re: Taxonomy Is Hard

#68
post #57

Earlier quoted context omitted.

> I, personally, personally cannot see this as a problem that can be solved. Maybe you are better. Both library science and cognitive linguistics have solutions for/discuss this problem.

I would say what they call 'composition by capability' is a theory of categorization, in fact similar to Wittgenstein's family resemblance theory. That said, it's not obvious that the underlying data model needs or should map to a theoretical model for how human cognition works. Maybe it should and it's worth considering the theoretical landscape before setting out, but there are other features that come into play. I…

You should definitely look into it, especially if you like pedantic arguments.

Re: Taxonomy Is Hard

#69
It's hard for everything in every way.

Biology. Everything is a fish or nothing is a fish. Trees don't exist. Tomatoes are fruits as are cucumbers, pumpkins, bell peppers, and most things we don't consider fruits. But all fruits are also vegetables. Strawberries are neither straw nor berries. Etc.

When they said the two hardest things in computer science was naming things and cache invalidation, it's partly because naming things is a hard problem in every discipline.

Re: Taxonomy Is Hard

#70
Been there and tried them all. One day you realise there isn’t a perfect approach and you must settle and compromise. I settled on project based [0]. When you notice too much repetition - and it happens more rarely than you may think - it’s time to simply consider a new project and a symlink. Not pure but is simple and practical.

[0] https://github.com/slowernews/hamster-system#hamster-folder-...

Post reply on HN