Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

I am endlessly fascinated with content tagging systems

twitter.com

Post reply on HN
Print view View original

171–180 of 279 posts

Re: I am endlessly fascinated with content tagging systems

#171

Post by [deleted] »

RussianCow wrote: post #147
dahdum wrote: post #71

I adore tagging systems and have worked on them in several different applications and implementations, but there are always pitfalls and trade offs, and it’s possible to bury yourself Nowadays I nearly always store the assigned tags as an integer array column in Postgres, then use the intarray extension to handle the arbitrary boolean expression searches like “((1|2)&(3)&(！5))”. I still have a tags table that stores…

The tradeoff here is that you lose the foreign key constraint, correct? So if you delete a tag, there is no way for the database to automatically remove all references to it. Or is there some way to do this now?

[deleted]

Re: I am endlessly fascinated with content tagging systems

#172

Post by tinco »

If you're at the point where you're adding hierarchies to your tags, I think you're fighting a losing battle. At that point, why not do what Google does and just make a BERT embedding. No way you're going to manually achieve the full extent of complexity of how humans group and describe things.

Re: I am endlessly fascinated with content tagging systems

#173

Post by jrochkind1 »

avgcorrection wrote: post #99
jrochkind1 wrote: post #14

> I can't find anything on how to design and implement anymore more than the barebones basics of a system. All of this stuff (horse/horses etc) is extensively discussed, maybe look under "taxonomy" or "ontology". Now, whether you want to use any of those solutions or not or find the discussion useful or not... if you aren't finding anything about it at all, you aren't looking in the right places. (I learned about it…

Librarians are the people that we (technologists) should learn from. But all I see is programmers trying to invent things from first principles.

Eh, as the librarian who wrote the post you're replying to... I am actually ambivalent.

I wish librarianship as a field and industry were more what I'd fantasize it should/could be, but it's not so much.

Re: I am endlessly fascinated with content tagging systems

#175

Post by redbar0n »

acchow wrote: post #174

Sounds like they are trying to embed the search semantics in the data storage. Why not treat search as a distinct problem?

Yes. Clicking a tag is like searching for a single word. The crux is that tags add useful metadata that may not be in the content it is supposed to tag. Maybe instead of modeling tags separately from content, metadata should simply be joined into the content itself at the end, and then searched using the same text search tools used for content?

Re: I am endlessly fascinated with content tagging systems

#176

Post by robocat »

stinkytaco wrote: post #97
pvg wrote: post #74

Earlier quoted context omitted.

(I learned about it in librarian school) As the rest of us learned during the first tagging boom, the librarian is the natural apex predator of tagging.

I've been a librarian for more than 15 years and I can only speak from personal experience when I say that I am the apex predator of nothing. Every once and a while I will get it in my head to systematize my personal knowledge base with a controlled vocabulary and ontology and I just fall on my face. I really want it for some twisted reason, though. Turns out LC subject headings -- for all their failures -- are prett…

> controlled vocabulary

Are you using English? English words can almost mean whatever you want them to. Perhaps design your own language that removes ambiguity. Probably requires a knowledge of philosophy to distinguish between say concrete and abstract, good luck.

Maybe start with correcting the ontology of: https://cuberule.com/ (which takes a geometric approach to defining food types).

Also perhaps decide whether you want to work top-down like a directory tree (or Dewey Decimal?): resulting in standard book classification issues. Or bottom up: resulting in conflicts and discrepancies - https://news.ycombinator.com/item?id=33254025

Re: I am endlessly fascinated with content tagging systems

#177

Post by openfuture »

googlryas wrote: post #83
openfuture wrote: post #78

Why twitter man.. these questions are clearly important but there is a space to discuss them https://matrix.to/#/#datalisp:matrix.org

I had to click like 5 links from that link in order to get to a site which requires me to sign in before allowing me to see the content. I still have no idea what I'm supposed to be seeing. And no idea what the connection between "datalisp" and content tagging systems is. Maybe that's why twitter man?

Yeah I am not a fan of the login walls and all that either but there is a reason that we should try to use free and open source software and currently matrix is the option that is convincing enough to use.

Datalisp.is the web of trust / semantic web / whatever. It doesn't exist outside my head currently but it also exists in lots of other heads (at least bits and pieces) so I believe we should manifest it.

Re: I am endlessly fascinated with content tagging systems

#178

Post by mceachen »

ggm wrote: post #114

Approximate date is the bugbear of photo tagging. EXIF and Dublin core and vendors can't agree what to do. Camera manufacturers don't care because at time of shot, date is fixed. It's archival, scanned and copied predigital work.

For what it's worth, ExifTool (and by extension, PhotoStructure) support 0 for month and day. The problem is that most all other applications won't see this as a valid date.

And I've struggled with how to covert this "fuzzy" date into something that sorts with other assets that _do_ have an exact date. Should they all live on the first of the month? In the middle of the fuzzy date range? Midnight? Noon?

Re: I am endlessly fascinated with content tagging systems

#179

Post by twobitshifter »

pessimizer wrote: post #54
kortex wrote: post #8

I think tag aliases are fine, but in my opinion, tags should not have hierarchies. That is just opening the can of ontology worms, and most systems are ill-equipped to deal with ontologies...including ontological systems. Tags are just dumb strings which label data. They are basically KeyValues, where the value is just always equal to True. We don't think of KVs as hierarchical unless they are explicitly a path strin…

Not having tag hierarchies doesn't fix the difficulty of classification, it just handwaves it away. There will always need to be (super)tags that are collections of other tags, where it is a bug for an item that has a particular tag to not also have another, related tag. The question should be how you're going to handle that, not if you're going to handle it, or you'll end up with a lot of broken tags of dubious usef…

This is just search with synonym analyzer / partial match. If you make the tag search dynamic, you’ll find the tags you’re looking for quickly.

Re: I am endlessly fascinated with content tagging systems

#180

Post by a1369209993 »

RussianCow wrote: post #147
dahdum wrote: post #71

I adore tagging systems and have worked on them in several different applications and implementations, but there are always pitfalls and trade offs, and it’s possible to bury yourself Nowadays I nearly always store the assigned tags as an integer array column in Postgres, then use the intarray extension to handle the arbitrary boolean expression searches like “((1|2)&(3)&(！5))”. I still have a tags table that stores…

The tradeoff here is that you lose the foreign key constraint, correct? So if you delete a tag, there is no way for the database to automatically remove all references to it. Or is there some way to do this now?

> So if you delete a tag, there is no way for the database to automatically remove all references to it.

I'm not sure about implementation/support in Postgres specifically, but in the general case of a column of tag bitfields, the database could easily maintain a global popcount (ie, "number of rows with this tag") and soft-delete flag for each tag, and clear any soft-deleted tags on (possibly-only-write-)access. When a soft-deleted tag reaches popcount == zero, it counts as garbage collected and can be reused for a new tag.

Post reply on HN