Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

I am endlessly fascinated with content tagging systems

twitter.com

Post reply on HN
Print view View original

151–160 of 279 posts

Re: I am endlessly fascinated with content tagging systems

#151

Post by VectorLock »

system2 wrote: post #128
VectorLock wrote: post #31

One example of an unexpectedly rich and deep tagging ontology is the Danbooru "Anime" image board [NSFW] https://danbooru.donmai.us/

I know this is not reddit. But why do you know even know this link and its tagging system...

I'm not scared away by things that might offend the puritanically inclined and I'm interested in ontologies and this is a fascinating one.

There was some drama about someone training a Stable-Diffusion-alike by ripping their dataset that brought it to my attention.

Re: I am endlessly fascinated with content tagging systems

#152

Post by matkoniecz »

xg15 wrote: post #82

I worked with the Wikipedia category system a few years ago, and you could see the problems with hierarchical tagging systems right in action back then. (Though it may have gotten better in the meantime) The system appeared simple: There were just two relations, "Article A is a member of category B" and "Category X is a subcategory of category Y". However, in practice, the community was using this system to represent…

I recently run into the same kind of problem in Wikidata.

https://www.wikidata.org/wiki/Wikidata_talk:WikiProject_Onto...

typical problem is of "light rail (Q1268865) is data visualization (Q6504956)" kind - this specific is fixed, but there are many similar

https://www.wikidata.org/wiki/Wikidata:Project_chat/Archive/...

https://www.wikidata.org/wiki/Wikidata:Project_chat#Ontology...

Re: I am endlessly fascinated with content tagging systems

#153

Post by matkoniecz »

maxbond wrote: post #100
crazygringo wrote: post #96

Earlier quoted context omitted.

I encountered the same problem a few years ago and indeed realized that using categories to understand what type of article a thing was (person? subject? event?) was utterly useless, for the reasons you describe. On the other hand, I discovered that infoboxes (the data in the top-right box on most pages) was generally extremely reliable, if frustrating to parse.

The infoboxes are created from a query to Wikidata, which you can query yourself! No scraping necessary! https://query.wikidata.org/ You'll want to learn SPARQL, but if you know SQL it's not so bad to pick up.

Wikidata is not solution at all.

I recently run into the same kind of problem in Wikidata.

https://www.wikidata.org/wiki/Wikidata_talk:WikiProject_Onto...

typical problem is of "light rail (Q1268865) is data visualization (Q6504956)" kind - this specific is fixed, but there are many similar

https://www.wikidata.org/wiki/Wikidata:Project_chat/Archive/...

https://www.wikidata.org/wiki/Wikidata:Project_chat#Ontology...

Re: I am endlessly fascinated with content tagging systems

#154

Post by matkoniecz »

emj wrote: post #112

Openstreetmap is map data that is basically coordinates with tags on them and relations between those tags. I guess this is true for most GIS software but there is very little 2D map data that can not be described in the OSM tagging model. You can never express everything with tags, you need stats and metadata on metadata, documentation and a strong heterogeneity which also need to be able to adapt to new ideas. http…

https://wiki.openstreetmap.org/wiki/Tagging_mailing_list ( https://lists.openstreetmap.org/pipermail/tagging/ ) is a fascinating, hilarious and interesting place.

Basically it is about an endless attempt to classify at least part of reality, in organically growing worldwide project based on bunch of passionate obsessive hobbyists with overly strong opinions.

With bonus of bunch of politics, confusion and passion.

https://wiki.openstreetmap.org/wiki/Overpass_API/Overpass_AP... is likely of interest.

Re: I am endlessly fascinated with content tagging systems

#155

Post by qwerty456127 »

This is crazily sad non-invasive (without embedding into the file body) tagging is not standardized across OSes and file systems. The only system to support tags I know is KDE/Dolphin/Baloo, outside KDE tagging seemingly is supported only by a handful of incompatible 3-rd party apps.

Sadly I don't expect much progress to happen in this area. Almost nobody cares about storing and organizing of files locally nowadays.

I hope it is going to be done some day or later (there isn't much to do: just standardize some xattrs and something like RDF schema to be used in an alternative FS stream + add support for these to the standard file management and search tools, this is orders of magnitude easier than implementing a new FS) but probably not soon - it would be a huge luck to get any resources allocated to this.

Re: I am endlessly fascinated with content tagging systems

#157

Post by meej »

edflsafoiewq wrote: post #18
jrochkind1 wrote: post #14

> I can't find anything on how to design and implement anymore more than the barebones basics of a system. All of this stuff (horse/horses etc) is extensively discussed, maybe look under "taxonomy" or "ontology". Now, whether you want to use any of those solutions or not or find the discussion useful or not... if you aren't finding anything about it at all, you aren't looking in the right places. (I learned about it…

Can you link some resources about it then?

This is a good basic overview, goes beyond tagging/indexing, was the textbook in LIS501 Information Organization and Access at UIUC-GSLIS (now the iSchool at Illinois) in 2006:

https://mitpress.mit.edu/9780262512619/the-intellectual-foun...

Controlled vocab standards:

https://www.niso.org/publications/ansiniso-z3919-2005-r2010

(this one is deprecated in favor the one that follows)

https://www.niso.org/schemas/iso25964

https://www.w3.org/2004/02/skos/

The book we used in my thesaurus construction class at UIUC:

https://www.alastore.ala.org/content/essential-thesaurus-con...

My favorite intro to semantic modeling with RDF/OWL/SPARQL:

http://workingontologist.org/

Topic Maps are dead but i still have a soft spot for them:

https://www.isotopicmaps.org/

I also recommend Heather Hedden, linked in jrockhind's post.

Re: I am endlessly fascinated with content tagging systems

#158

Post by jrumbut »

pessimizer wrote: post #121
xg15 wrote: post #82

I worked with the Wikipedia category system a few years ago, and you could see the problems with hierarchical tagging systems right in action back then. (Though it may have gotten better in the meantime) The system appeared simple: There were just two relations, "Article A is a member of category B" and "Category X is a subcategory of category Y". However, in practice, the community was using this system to represent…

> I'm sure there are sophisticated ontological systems which would allow users to specify all those different relationships separately. I'm also pretty sure that users would become sloppy after a short time or would disagree which particular relationship to use in a particular situation I think the problem is allowing users to freely tag, then. There should be easily accessed guidelines about how each tag should be u…

The fundamental tension in tagging systems, to me, is whether tagging is a feature the software offers to the user or a task the user performs to assist the software.

In the first case, you want freewheeling and tolerate ontological inconsistencies because you want to offer flexibility to users and will capture hard to quantify emergent benefits (some made up examples: "try the tag user233-favorite, I keep discovering awesome articles!", "the physicist-needed tag has highlighted a lot of misinformation surrounding quantum physics and relativity"). People use it to the extent it is useful.

The other way, with formal semantics, governance (which you made some very wise points about), etc allows the software to reply to queries like "19th-century + Missouri + humorists" in a performant and authoritative way. It's not really a feature so much as it is a way to enable other features.

Re: I am endlessly fascinated with content tagging systems

#159

Post by layer8 »

xg15 wrote: post #82

I worked with the Wikipedia category system a few years ago, and you could see the problems with hierarchical tagging systems right in action back then. (Though it may have gotten better in the meantime) The system appeared simple: There were just two relations, "Article A is a member of category B" and "Category X is a subcategory of category Y". However, in practice, the community was using this system to represent…

There are only two kinds of relation here, “subset of” and “instance of” (aka “element of”, type-token).

The category-category relations are intended to always be a subset relation. The article-category relations are intended to always be an instance-of relation.

- "19th century American writers" is a subset of "American writers“.

=> Both are a category, so no problem.

- “Novelists” is a subset of “Writers”.

=> Both are a category, so no problem.

- “Writer(s)” is an instance of “Occupation”.

=> Here the problem is that “Writers” is a category. It would be okay if it was an article “Writer (occupation)”.

- “Writers” is an instance of “Categories with more than 100 entries".

=> Here, again, the problem is that “Writers” is a category, and having an instance-of relation between categories is not an intended/supported use-case.

This could conceivably be solved by supporting an instance-of relation between categories, in addition to the existing subset (subcategory) relation. It could be called a meta-category relation. Then you could have the category of occupation categories.

Another way to put this is that categories have to be typed: a category contains either (just) articles, or it contains (just) categories. Subcategories then must match the type of their supercategories and correspondingly must contain either articles or categories.

Basically, Wikipedia’s type system is not expressive enough to allow everything people would want to express in it.

Re: I am endlessly fascinated with content tagging systems

#160

Post by TazeTSchnitzel »

VectorLock wrote: post #31

One example of an unexpectedly rich and deep tagging ontology is the Danbooru "Anime" image board [NSFW] https://danbooru.donmai.us/

There is a safe-for-work, or at least safer-for-work version of the site: https://safebooru.donmai.us/

(It is of course based on the tagging system: every post is tagged by its "safeness" level.)

Post reply on HN