Json-wikipedia
github.com
Json-wikipedia
1–10 of 17 posts
Re: Json-wikipedia
#2What about DBPedia http://dbpedia.org/page/The_Shining_(film) ?
That's available as JSON, and much larger community/effort behind it.
Re: Json-wikipedia
#3What about DBPedia http://dbpedia.org/page/The_Shining_(film) ? That's available as JSON, and much larger community/effort behind it.
There's a much larger community/effort behind DBPedia than Wikipedia?
Or are you comparing DBPedia to this utility that converts Wikipedia data dumps?
Edit: looks like I need to do some more research into DBPedia. It sounds like that community has created a dataset that categorizes Wikipedia article (e.g. "things" and "facts")
Re: Json-wikipedia
#4What about DBPedia http://dbpedia.org/page/The_Shining_(film) ? That's available as JSON, and much larger community/effort behind it.
Correct me if I am wrong, but I do not think that DBPedia includes the full Wikipedia articles (only extended abstracts). So I believe that json dumps of DBPedia and Wikipedia dumps produced using this tool are two different things.
Re: Json-wikipedia
#5What about DBPedia http://dbpedia.org/page/The_Shining_(film) ? That's available as JSON, and much larger community/effort behind it.
In my understanding, DBpedia is a project for extracting data mainly from Wikipedia infoboxes (not whole data dumps) by collaboratively creating rules for converting data into cleaner schema that enables to perform a query such as SPARQL.
I think this is a project that directly converts wikipedia dump xml to JSON for easier manipulation, which differs from DBpedia.
Re: Json-wikipedia
#6So, does it parse the markdown of the text as well? I could not quite get it from the documentation. It would be quite exciting if it could - the markdown is notoriously complex, and creates a lot of problem if all you want is te body of text (e.g. for training a machine learning algorithm).
Re: Json-wikipedia
#7So, does it parse the markdown of the text as well? I could not quite get it from the documentation. It would be quite exciting if it could - the markdown is notoriously complex, and creates a lot of problem if all you want is te body of text (e.g. for training a machine learning algorithm).
I doubt it (though I haven't tested it). Wikipedia's API though lets you get an HTML version of a page.
Re: Json-wikipedia
#8So, does it parse the markdown of the text as well? I could not quite get it from the documentation. It would be quite exciting if it could - the markdown is notoriously complex, and creates a lot of problem if all you want is te body of text (e.g. for training a machine learning algorithm).
Wikipedia articles are not written in markdown but in wikicode. Wikicode is much more complex to parse, but allows more complex formatting. You may be looking for the word 'markup' which is a general term that included markdown, HTML, wikicode, bbcode etc.
I doubt this library parses the wikicode. What format would it parse it into? As matthewarkin said, there is a Wikipedia API to convert wikicode into HTML.
Re: Json-wikipedia
#9Why would I need the dump in JSON? XML is for sure uglier but at least there are efficient SAX parsers.
Re: Json-wikipedia
#10Why would I need the dump in JSON? XML is for sure uglier but at least there are efficient SAX parsers.
I would like to know this as well. What is gained by using this tool... converting to JSON?