Live data from Hacker News

Wikipedia is now drawing facts from the Wikidata repository

gigaom.com

11–20 of 82 posts

Re: Wikipedia is now drawing facts from the Wikidata repository

#11
post #6

It's too bad that all of these grand visions and truly positive developments for humanity are locked inside of an exclusionary, elitist organization that is governed by deletionism and will take any contribution you might possibly make and CRUSH IT LIKE A BUG.

> deletionism

you might enjoy Deletionpedia: http://www.deletionpedia.dbatley.com/w/index.php?title=Main_...

Re: Wikipedia is now drawing facts from the Wikidata repository

#12
post #4

OT: I was using Wikipedia the other day and it occurred to me how primitive it is to have all the inner links to other Wikipedia articles defined manually, surely these should have been automated by now (i.e., marking a word or two would link you to the relevant article).

The disambiguation could be challenging. (Does "The Sun" link to our closest star or the tabloid in the UK.)

It would be a fun project to try and determine the correct link based on the context.

Re: Wikipedia is now drawing facts from the Wikidata repository

#13
post #5
post #4

OT: I was using Wikipedia the other day and it occurred to me how primitive it is to have all the inner links to other Wikipedia articles defined manually, surely these should have been automated by now (i.e., marking a word or two would link you to the relevant article).

I would venture there's a way to make "concepts" and "entities" become linkable automatically based on existing articles, but that would mean a bit of engineering. I.E. A name, product or academic field. But then there's going to be a high number of links to articles that haven't been created yet or deleted/merged etc... in cases of lack of notability.

I don't know if it has that exact feature or not, but Semantic Mediawiki has a lot of extensions to base Mediawiki that involve working with data at a semantic level.

http://semantic-mediawiki.org/wiki/Help:Introduction_to_Sema...

If not, doing that should be possible using an NLP library that does NER. Along with heuristics, one could use the list of currently existing articles as a seed.

Edit: Of course, if all you're trying to do is link to existing pages, then you don't use the set of existing pages as a seed, you just use them as the list. But if you're trying to extract "entities" that don't have WP pages yet, then you'd still want to fall back to other NER techniques, which include various heuristics and what-not. Whether or not there would be an value in that is an open question, I suppose.

Re: Wikipedia is now drawing facts from the Wikidata repository

#14
post #4

OT: I was using Wikipedia the other day and it occurred to me how primitive it is to have all the inner links to other Wikipedia articles defined manually, surely these should have been automated by now (i.e., marking a word or two would link you to the relevant article).

The disambiguation could be challenging. (Does "The Sun" link to our closest star or the tabloid in the UK.) It would be a fun project to try and determine the correct link based on the context.

Just make the link to the disambiguation page, if there is one? Otherwise, make it a special link that doesn't go anywhere directly, but uses some javascript/CSS to raise a dialog when clicked, that gives you the different choices?

Re: Wikipedia is now drawing facts from the Wikidata repository

#15
Wikidata is interesting ... it seems to be a normal MediaWiki install with the custom-developed "WikiBase" extension: https://www.mediawiki.org/wiki/Extension:Wikibase

It'd be nice if the WikiMedia projects had a proper GitHub presence - it's hard to get a sense how plausible a self-hosted version of this is.

Re: Wikipedia is now drawing facts from the Wikidata repository

#16
post #5

Earlier quoted context omitted.

I would venture there's a way to make "concepts" and "entities" become linkable automatically based on existing articles, but that would mean a bit of engineering. I.E. A name, product or academic field. But then there's going to be a high number of links to articles that haven't been created yet or deleted/merged etc... in cases of lack of notability.

I don't know if it has that exact feature or not, but Semantic Mediawiki has a lot of extensions to base Mediawiki that involve working with data at a semantic level. http://semantic-mediawiki.org/wiki/Help:Introduction_to_Sema... If not, doing that should be possible using an NLP library that does NER. Along with heuristics, one could use the list of currently existing articles as a seed. Edit: Of course, if all you…

Good resource! And I think your idea of linking to the disambiguation page makes sense, but there may be a way to infer the correct article from the list of links based on the context of the text in the linking article.

Re: Wikipedia is now drawing facts from the Wikidata repository

#17
post #6

It's too bad that all of these grand visions and truly positive developments for humanity are locked inside of an exclusionary, elitist organization that is governed by deletionism and will take any contribution you might possibly make and CRUSH IT LIKE A BUG.

This is a good example for Paul Graham's "middlebrow dismissal" data set. Generic negative comment about the subject of the article that could be copy-pasted into any article matching a "Wikipedia" string search by a bot.

Re: Wikipedia is now drawing facts from the Wikidata repository

#18
post #16

Earlier quoted context omitted.

I don't know if it has that exact feature or not, but Semantic Mediawiki has a lot of extensions to base Mediawiki that involve working with data at a semantic level. http://semantic-mediawiki.org/wiki/Help:Introduction_to_Sema... If not, doing that should be possible using an NLP library that does NER. Along with heuristics, one could use the list of currently existing articles as a seed. Edit: Of course, if all you…

Good resource! And I think your idea of linking to the disambiguation page makes sense, but there may be a way to infer the correct article from the list of links based on the context of the text in the linking article.

Yeah, there's that as well.

Also, if you're interested in that sort of thing, two other projects you might find interest are:

http://stanbol.apache.org

and

http://uima.apache.org

Both involve extracting semantic meaning from unstructured data. It's pretty cool stuff.

Re: Wikipedia is now drawing facts from the Wikidata repository

#19
Good. For quite some time, census data was incorporated as a one-time substitution into geographical articles by a bot. With this new development, I suppose that census data can be incorporated as a "transclusion" that is updated automatically either on schedule or on demand.

Re: Wikipedia is now drawing facts from the Wikidata repository

#20
post #6

It's too bad that all of these grand visions and truly positive developments for humanity are locked inside of an exclusionary, elitist organization that is governed by deletionism and will take any contribution you might possibly make and CRUSH IT LIKE A BUG.

> deletionism you might enjoy Deletionpedia: http://www.deletionpedia.dbatley.com/w/index.php?title=Main_...

Very interesting :)

I can see why most were deleted, I chose to view a few pages at random and they were:

- a really non-notable musician (probably self-promotion)

- a hoax ("The Independent City-State of Sonora")

- a game guide to Super Smash Brawl, this is probably something that should have gone on a blog / GameFAQs

- a witty bio about a non-notable person (probably self-bio, or a friend's)

I believe the four of them were deservedly well deleted. Two of the cases, the musician and the game guide, should be hosted elsewhere (personal blog or website).

Post reply on HN