Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

I am endlessly fascinated with content tagging systems

twitter.com

Post reply on HN
Print view View original

191–200 of 279 posts

Re: I am endlessly fascinated with content tagging systems

#191

Post by derefr »

dahdum wrote: post #71

I adore tagging systems and have worked on them in several different applications and implementations, but there are always pitfalls and trade offs, and it’s possible to bury yourself Nowadays I nearly always store the assigned tags as an integer array column in Postgres, then use the intarray extension to handle the arbitrary boolean expression searches like “((1|2)&(3)&(！5))”. I still have a tags table that stores…

Read perf would be good in isolation, sure; but what about the lock contention over those rows due to the frequent writes to the tags column? Usually "taggings" have a much higher rate-of-change than the objects they tag. If you're not at least keeping a thing_taggings has-one table (thing bigserial, tags intarray) separate from your things table, I could see this degrading performance for any query that wants to touch the table.

Re: I am endlessly fascinated with content tagging systems

#192

Post by ethn »

Generally you either use Latent Dirichlet Allocation, exact tags, or a mixture of both. I structure the metric space to weigh exact tags greater than LDA—-whereas you can then create two more classes in that LDA space, of the heavier similar tags and then the description.

Re: I am endlessly fascinated with content tagging systems

#193

Post by OJFord »

errantmind wrote: post #110

Instagram's tagging system was actually really effective at categorizing content and discovery because each hashtag was treated as a node in a (giant) graph, where each node has multiple properties, including post count (number of posts using a tag), 'velocity' (number of posts using a particular tag per unit time), etc. I could write up a big post about it as I made a study of it in when I created a web app for find…

I'd also be interested in reading about that - in particular are things like 'post count' computed and stored (i.e. not normalised)? How do you cope with keeping such analytics current as the source changes?

(When oh when will postgres get automatic and incremental materialised views?)

Re: I am endlessly fascinated with content tagging systems

#194

Post by papito »

aaviator42 wrote: post #37

A few months ago I worked on some proof-of-concept code for searching tagged data: https://github.com/aaviator42/Cha I now work full-time in a role where part of my duties is designing a content tagging system and its search functionalities. It's very interesting and fun! Lots of puzzles. How do you weigh different tags? How do you do fuzzy searching ('city' should match with plural ('cities'), misspellings ('citys')…

Seems to me like a lot of these are solved by a dedicated search engine?

I see it in this business all the time how people try to reinvent the wheel and end up writing their own search engine, thinking it's just another small in-house project, but they quickly run into the difficult problems like, well, these.

Re: I am endlessly fascinated with content tagging systems

#195

Post by pkd »

aaviator42 wrote: post #37

A few months ago I worked on some proof-of-concept code for searching tagged data: https://github.com/aaviator42/Cha I now work full-time in a role where part of my duties is designing a content tagging system and its search functionalities. It's very interesting and fun! Lots of puzzles. How do you weigh different tags? How do you do fuzzy searching ('city' should match with plural ('cities'), misspellings ('citys')…

Nice. Your Cha project has the beginnings of a search engine with cosine similarity, even though I don't think you intend to take it there. Tagged items are like a precomputed inverted index, and the matching is a search on that index.

Re: I am endlessly fascinated with content tagging systems

#196

Post by SicSemperUranus »

errantmind wrote: post #110

Instagram's tagging system was actually really effective at categorizing content and discovery because each hashtag was treated as a node in a (giant) graph, where each node has multiple properties, including post count (number of posts using a tag), 'velocity' (number of posts using a particular tag per unit time), etc. I could write up a big post about it as I made a study of it in when I created a web app for find…

> in favor of other methods for classification and discovery

Aka the toxic engagement trap. I miss the days when unfathomable AI didn't dictate what's popular.

Re: I am endlessly fascinated with content tagging systems

#197

Post by robocat »

subpar wrote: post #77

I've done this professionally in a couple different settings, from building topic classifiers for news events (it is sometimes hard to know when one news event should stop and another start) to creating tagging systems for audio recordings of group conversations (where topics often merge in and out of each other, often within a single sentence). I'm currently working on classifying non-speech, non-musical sound and i…

The Google list is interesting. It only has main category and subcategory. Causes problems like “SCIFI WEAPON SCIWeap = Lightsabers, exotic sci-fi weapons. Not a 'blaster' which would go in LASERS-GUN.” because they created another main category LASERS for the desired four extra sub-subcategories (hmmmm, how should that be spelled? Categorically another conundrum.)

Re: I am endlessly fascinated with content tagging systems

#199

Post by kortex »

pessimizer wrote: post #54
kortex wrote: post #8

I think tag aliases are fine, but in my opinion, tags should not have hierarchies. That is just opening the can of ontology worms, and most systems are ill-equipped to deal with ontologies...including ontological systems. Tags are just dumb strings which label data. They are basically KeyValues, where the value is just always equal to True. We don't think of KVs as hierarchical unless they are explicitly a path strin…

Not having tag hierarchies doesn't fix the difficulty of classification, it just handwaves it away. There will always need to be (super)tags that are collections of other tags, where it is a bug for an item that has a particular tag to not also have another, related tag. The question should be how you're going to handle that, not if you're going to handle it, or you'll end up with a lot of broken tags of dubious usef…

> It means that you're building a knowledge base.

That's precisely the problem. You started with tags and now you are building a knowledge base. You wanted a banana and now you have a gorilla holding a banana and the whole jungle. If you want to build a knowledge base, use links/URIs/ontologies.

You'll find out the moment you add cycles the algorithms get way more intense. And then once you have stronger algos, you want more search power. Next thing you know you are bikeshedding about things like "apple" is both "fruit" and "tech company" so you need tags-of-tags etc. Just build a knowledge base if you need a knowledge base. Otherwise tags are just a way to do faceted search.

They aren't mutually exclusive, either.

OpenTelemetry, for example, has both tags and references.

Re: I am endlessly fascinated with content tagging systems

#200

Post by Leftium »

counttheforks wrote: post #13

Anyone have a suggestion for a tagging filesystem that is maintained? Or if not a filesystem, something that at least works? I still feel like this is the best way to organize personal photos and media, and while https://www.tagsistant.net/ is pretty good it hasn't been updated in 6 years and is fairly buggy.

Directory Opus embeds labels into the file system:

https://www.gpsoft.com.au/help/opus12/index.html#!Documents/...

Post reply on HN