Live data from Hacker News

I am endlessly fascinated with content tagging systems

twitter.com

71–80 of 279 posts

Re: I am endlessly fascinated with content tagging systems

#71
I adore tagging systems and have worked on them in several different applications and implementations, but there are always pitfalls and trade offs, and it’s possible to bury yourself

Nowadays I nearly always store the assigned tags as an integer array column in Postgres, then use the intarray extension to handle the arbitrary boolean expression searches like “((1|2)&(3)&(!5))”. I still have a tags table that stores all the metadata / hierarchy / rules, but for performance I don’t use a join table. This has solved most of my problems. Supertags just expand to OR statements when I generate the expression. Performance has been excellent even with large tables thanks to pg indexing.

Re: I am endlessly fascinated with content tagging systems

#73
I set out building my first full-stack webapp [0] to make a custom theme-based tagging/organizational system for musical ideas. I did not initially realize all the hairy design choices inherent in this domain, but have found it humbling and educational.

Remaining features to be implemented include in-app audio recording, editing, and custom labeling outside of the main tree structured organizational system.

I'd appreciate any thoughts or suggestions if anyone cares to take a look!

[0] https://www.soundseeker.app/

Re: I am endlessly fascinated with content tagging systems

#74

> I can't find anything on how to design and implement anymore more than the barebones basics of a system. All of this stuff (horse/horses etc) is extensively discussed, maybe look under "taxonomy" or "ontology". Now, whether you want to use any of those solutions or not or find the discussion useful or not... if you aren't finding anything about it at all, you aren't looking in the right places. (I learned about it…

(I learned about it in librarian school)

As the rest of us learned during the first tagging boom, the librarian is the natural apex predator of tagging.

Re: I am endlessly fascinated with content tagging systems

#75
This is the reason the Semantic Web never took off—people on the internet can't even agree on what a "sandwich" is, let alone the exact hierarchy of ontology.

This is an area where large language models have a role to play—whatever you're hoping to achieve with user-generated tags can probably be achieved with ML-powered associations or navigation. And the potential benefit is that it could be tailored to each user—so you're only surfacing "Hot Dogs" when certain users click "Sandwich."

Re: I am endlessly fascinated with content tagging systems

#76

Earlier quoted context omitted.

> How do you program the system so that 'hotdog' is not matched with 'hot' and 'dog'? That sounds like a very good use case for word embeddings.

How do you deal with "hotdog" possibly being a noun (several meanings), or proper noun (several meanings), or verb, or interjection?

e621 frequently has to deal with characters with the same name, or an artist with the same name as a character. they just make ambiguous tags have a special syntax. so if bob was an artist, but also had a character named bob, it would just be bob_(bob) for the character and bob_(artist) for the artist. and if someone tried to tag something as just “bob” they would be told to be more specific. searching for all bobs can be done with bob_(*).

so hotdog could have hotdog_(food), hotdog_(interjection), and hot dogs (the animal) would be two tags: hot and dog.

it’s not the cleanest solution, but it works well enough.

Re: I am endlessly fascinated with content tagging systems

#77
I've done this professionally in a couple different settings, from building topic classifiers for news events (it is sometimes hard to know when one news event should stop and another start) to creating tagging systems for audio recordings of group conversations (where topics often merge in and out of each other, often within a single sentence).

I'm currently working on classifying non-speech, non-musical sound and it can be useful to piggyback on an existing knowledge system, though they tend to be industry-specific. As an example, Google's ontology for sound identification [1] is a nice starting point for general classification, whereas the taxonomy [2] used by the audio post-production industry (sound effects, foley, etc) is structurally quite different (which isn't surprising, but it sure is fun!). From a totally different field (electro-acoustic composition), the work of Michel Chion and Pierre Schaeffer [3] add psychoacoustic elements to more traditional measurable characteristics, i.e. how the sound is perceived and comprehended is just as important as its medium of travel and its source. It is helpful to see what others have done before you so you can pick and choose elements of their work to incorporate into your own.

1: https://github.com/audioset/ontology

2: https://docs.google.com/spreadsheets/d/1b2UhKpcOAE-jd1edOsxC...

3: [big pdf!] https://monoskop.org/images/0/01/Chion_Michel_Guide_To_Sound...

Re: I am endlessly fascinated with content tagging systems

#79

Earlier quoted context omitted.

I will paraphrase this to avoid a callout, but "no regenerating limbs those arms are toast sorry QA despises them" is not a useful tag . (This is a mild example, I've seen far worse) First, it is a single-use tag. Tags are for categories , not solo entries. Solo entries explode the tagspace to no good end. Second, that expression belongs in the summation of the work, or just about anywhere else. Tags are for other pe…

I think you're missing the point that, in AO3 specifically, tags are not solely metadata. Tags are also artistic expression in the context of AO3 . That's the thing. AO3 doesn't function like the Library of Congress, and there are no librarians that are independently assigning categories to fanfic. An author can choose to opt out of tags entirely, and people cannot put tags on other people's fanfic even if it's relev…

> An author can choose to opt out of tags entirely, and people cannot put tags on other people's fanfic even if it's relevant and would benefit that work's findability

Curious about how this doesn't render the entire system near-useless? In my experience with other sites with user-generated content that allow tagging, this decision always makes the whole system way worse, because the OP alone is almost never going to be aware of all possible tags that are applicable to whatever it is they posted, and will instead just take the first 3-5 words that pop into their head and stick those in the tags field. The end result is a tagging system that barely works; you can search for a tag but you'll miss tons of stuff, and you can filter out a tag but you'll still see tons of stuff in that category. And if you ever find a hyper-specific tag you really enjoy it'll only have like 5 items in it even if there are hundreds or thousands it could be applicable to.

Don't get me wrong, the wiki-style approach of just letting anyone edit tags has its own issues, but it does at least result in tags on everything being at least mostly complete, and actually useful for finding what you want (or filtering out things you don't want).

Re: I am endlessly fascinated with content tagging systems

#80
post #56
post #51

Earlier quoted context omitted.

It's lower effort to make a stream of consciousness post one sentence at a time, and as a bonus, there's a built in audience / discovery network where they're posting.

Lower effort for whom? Back when I were a lad, we were told to write so that our readers did not have to work to understand us. The point of writing is to be understood. Old man yells at cloud.

I think it's helpful to keep in mind that with most of examples that get shared around, the choice for the author was not a string of tweets vs blog post, but rather a string of tweets vs not sharing at all.
Post reply on HN