Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

I am endlessly fascinated with content tagging systems

twitter.com

Post reply on HN
Print view View original

211–220 of 279 posts

Re: I am endlessly fascinated with content tagging systems

#211

Post by dahdum »

RussianCow wrote: post #147
dahdum wrote: post #71

I adore tagging systems and have worked on them in several different applications and implementations, but there are always pitfalls and trade offs, and it’s possible to bury yourself Nowadays I nearly always store the assigned tags as an integer array column in Postgres, then use the intarray extension to handle the arbitrary boolean expression searches like “((1|2)&(3)&(！5))”. I still have a tags table that stores…

The tradeoff here is that you lose the foreign key constraint, correct? So if you delete a tag, there is no way for the database to automatically remove all references to it. Or is there some way to do this now?

Yes, but that’s easily handled with a trigger. My first implementation actually had a regular join table of items_tags which used a trigger to update the items.tags intarray. Wasn’t super performant but let us use our existing templates for 1-many to implement the UI.

Nowadays you can just use a tagging component with integrated search for the UI.

Re: I am endlessly fascinated with content tagging systems

#212

Post by dahdum »

srcreigh wrote: post #124
dahdum wrote: post #71

I adore tagging systems and have worked on them in several different applications and implementations, but there are always pitfalls and trade offs, and it’s possible to bury yourself Nowadays I nearly always store the assigned tags as an integer array column in Postgres, then use the intarray extension to handle the arbitrary boolean expression searches like “((1|2)&(3)&(！5))”. I still have a tags table that stores…

Do you index arrays? What index type is that? Any tips? I’ve used array column in PG before, haven’t indexed arrays though.

Yes, it’s explained in the intarray doc here. GiST is the one I use, but as it states GIN should be faster on reads. I haven’t really thought about that in many years, I should run some perf tests.

https://www.postgresql.org/docs/9.1/intarray.html

Re: I am endlessly fascinated with content tagging systems

#213

Post by dahdum »

langsoul-com wrote: post #210
dahdum wrote: post #71

I adore tagging systems and have worked on them in several different applications and implementations, but there are always pitfalls and trade offs, and it’s possible to bury yourself Nowadays I nearly always store the assigned tags as an integer array column in Postgres, then use the intarray extension to handle the arbitrary boolean expression searches like “((1|2)&(3)&(！5))”. I still have a tags table that stores…

I just made tags nosql, it's stored as json data in pg with a user set to each tag. I think the relation was that each user could have mutlitple posts that each contain multiple tags. Where each tag is global with an user ids stored with the user Id as the key. Or something approximatating that. Mainly to not need insane queries for many to many relationships. O(1) baby.

Can you search by tag expressions like “((1|2)&(!3))” with your method? I’m not understanding it completely. I often use json_data but I didn’t consider tagging methods because of the overhead it has.

Re: I am endlessly fascinated with content tagging systems

#214

Post by kixiQu »

errantmind wrote: post #204
kixiQu wrote: post #202

Earlier quoted context omitted.

I went and got my laptop to type up a reply to this: Instagram's tagging system was and is atrocious in combination with their discovery mechanisms and the incentives they create. A real example, this has been true for years: I want to look at pictures of Jennifer Lawrence's makeup because she, like me, has hooded eyes and that makes useful reference. I go to instagram imagining that I will find fan accounts posting…

Isolated hashtags on IG were often not useful for finding specific content. To find what you want you'd need an interface to a page that showed posts across multiple hashtags which they didn't provide. For example, posts that contain both #jenniferlawrence and #hoodedeyes would be what you are looking for, and if you interacted with enough posts containing both of those tags you would end up seeing the content you wa…

> For example, posts that contain both #jenniferlawrence and #hoodedeyes would be what you are looking for

I mean, not to overwork it, but in this example, this isn't accurate: I just want pictures of Jennifer Lawrence, not the much smaller slice of content where the poster had her particular eye shape in mind. Also, I don't want to only "end up seeing" pictures of Jennifer Lawrence in my main feed, I want to be able to go find them when I want them. Instagram's ontology is designed to not make this possible, because everything about it is meant to facilitate tube-feeding, and that is why I feel so strongly that it is trash.

(We could also talk about how similar phenomena manifested elsewhere: the #goth tag on Tumblr in my Tumblr days was unusable because of the quantity of non-goths looking at a lowkey photo and deciding that was the best descriptor – so the actual #goth content was found in .... #gothgoth. And if you're wondering if this was discovered and subsequently chased out to #gothgothgoth, You Have Understood The Problem)

Re: I am endlessly fascinated with content tagging systems

#215

Post by andrewflnr »

maratc wrote: post #115
xg15 wrote: post #82

I worked with the Wikipedia category system a few years ago, and you could see the problems with hierarchical tagging systems right in action back then. (Though it may have gotten better in the meantime) The system appeared simple: There were just two relations, "Article A is a member of category B" and "Category X is a subcategory of category Y". However, in practice, the community was using this system to represent…

The problem might not be with hierarchical tagging systems, but with the specific hierarchical tagging system they use at Wikipedia. Imagine another system with the following categories: * People:ByOccupation:Creative:Writers * Time:CommonEra:ByCentury:19 * Location:Earth:Americas:NorthAmerica:USA In this scheme of things, e.g. Mark Twain would be tagged with all three. "19th century American writers" (which includes…

Suppose you have someone who did a bunch of writing in America, then moved to Europe and became famous as an inventor there. Under your proposal, this person has both Location:Europe and Location:USA, and both Occupation:Writer and Occupation:Inventor. They therefore show up for queries for European writers and American inventors, neither of which was intended; I bet we can come up with situations where the false positives are even worse. The presence of those tags have to be interpreted in light of each other.

If you do this naively I think it's pretty clear you've either sacrificed expressivity or made the system a LOT more complicated/harder to understand. At best you end up with some kind of product structure in (what is no longer just) the set of tags on an article. You can think of explicating an implicit product structure in joining "American" and "Writer" in the same object. But I think if you've started talking about compound tags, you're really talking about something other than a tagging system.

Re: I am endlessly fascinated with content tagging systems

#216

Post by youainti »

xg15 wrote: post #82

I worked with the Wikipedia category system a few years ago, and you could see the problems with hierarchical tagging systems right in action back then. (Though it may have gotten better in the meantime) The system appeared simple: There were just two relations, "Article A is a member of category B" and "Category X is a subcategory of category Y". However, in practice, the community was using this system to represent…

> m sure there are sophisticated ontological systems which would allow users to specify all those different relationships separately. I'm also pretty sure that users would become sloppy after a short time or would disagree which particular relationship to use in a particular situation...

You might be interested in Snowmed CT, a way to describe medical concepts. It does something rather similar.

Re: I am endlessly fascinated with content tagging systems

#217

Post by hamasho »

When I started creating a simple blog system as a newbie developer, I needed to design its category/tagging system. Then I was surprised by the lack of good resources on how to design such basic features. I just wanted to know several design patterns and their pros and cons, but I couldn't find any, so I ended up designing my own crappy system.

I hope someone wrote articles on it with actual DB schemas.

Re: I am endlessly fascinated with content tagging systems

#218

Post by VectorLock »

thrdbndndn wrote: post #170
VectorLock wrote: post #31

One example of an unexpectedly rich and deep tagging ontology is the Danbooru "Anime" image board [NSFW] https://danbooru.donmai.us/

Yeah, danbooru or similar image boards basically have all the things talked in this tweet thread. They have tag aliases, meta-tags and so-called "tag implications". The last one is basically sub-tags but with more flexibility and dead simple to implement: if A implicates B, then tagging an image with A will automatically tag it with B. So you can tag "American Male Novelist", and then the system will automatically ad…

How much content they've actually put in their tagging system is just as interesting as how the tagging system works.

Re: I am endlessly fascinated with content tagging systems

#219

Post by c7b »

Tags are arguably superior to folders for organising files, unfortunately the major OS don't seem to agree. I'm using the same (expanded, adapted) folder structure for all my files since I got my first computer, and it's survived multiple OS migrations, being synced between multiple devices with different form factors, multiple changes in life circumstances (school, undergrad, postgrad, work),... I love tags and I've used them in some parts (eg in my old mp3 collection, for academic papers, for Anki flash cards) and I'd love to use a (simple and dumb, not rich enough to enable set theory paradoxes) tagging system to organise my files instead.

However, my experience has left me convinced that the only truly long-term solution for your personal data are flat files sitting on your hard drive inside a simple hierarchical folder structure. Anything else is likely going to rot at some point, after a system change, after some BigTech decides they want to use something else, after a start-up disappears, or it's going to keep you locked into some walled garden. Unless there's something I've missed, if so please let me know.

Re: I am endlessly fascinated with content tagging systems

#220

Post by errantmind »

kixiQu wrote: post #214
errantmind wrote: post #204

Earlier quoted context omitted.

Isolated hashtags on IG were often not useful for finding specific content. To find what you want you'd need an interface to a page that showed posts across multiple hashtags which they didn't provide. For example, posts that contain both #jenniferlawrence and #hoodedeyes would be what you are looking for, and if you interacted with enough posts containing both of those tags you would end up seeing the content you wa…

> For example, posts that contain both #jenniferlawrence and #hoodedeyes would be what you are looking for I mean, not to overwork it, but in this example, this isn't accurate: I just want pictures of Jennifer Lawrence, not the much smaller slice of content where the poster had her particular eye shape in mind. Also, I don't want to only "end up seeing" pictures of Jennifer Lawrence in my main feed, I want to be able…

Right, but it was never meant to specialize in user-directed filtering and discovery to the level you are describing.

To your second point, every social context that gets 'cool' eventually gets LCDed into mediocrity (even HN). You have to outrun the noise, as you described in your #gothgothgoth example

Post reply on HN