Live data from Hacker News

The Parser that Cracked the MediaWiki Code

dirkriehle.com

31–33 of 33 posts

Re: The Parser that Cracked the MediaWiki Code

#31
post #15

This is a breakthrough and a welcome one. From a end user point of view, it has a couple major implications. First, I believe this reveals the complexity of the parser, which implies a complex syntax, which implies a complex user interface as felt by end users. A more complex the user interface may make it harder it is to attract new editors, although it's unclear (to me) if that is a fact. Second, having an AST repr…

First, I believe this reveals the complexity of the parser, which implies a complex syntax, which implies a complex user interface as felt by end users.

That's the case to some extent, but the opposite is also the case to some extent. Some of the difficulty of parsing is because "ease of human use" has been a much higher priority than "ease of parsing" when discussing syntax, which leads to some constructs that aren't easy to parse with typical CFG-type parsing approaches. It's also designed to be very lenient to ordering and common errors, much like a modern non-strict HTML parser, which makes hand-writing the syntax more friendly and forgiving, but with a tradeoff that the parser has to be more complex, because it doesn't have the luxury of just returning a parse error.

Re: The Parser that Cracked the MediaWiki Code

#32
post #23
post #18

Earlier quoted context omitted.

Markdown...ugh. Let's just stick to DokuWiki or Mediawiki syntax for everything, please. If you need something more advanced than that, you should be using LaTeX. Actually, it'd be cool to build a working MediaWiki + Markdown => LaTeX converter....in something like Python.

there is markdown2pdf written Haskell, which seems to have XeTeX as an intermediary step. Personally I'd be happy to see any markup language becoming the default, regardless which one it is. Having a proper grammar would be a bonus.

"Personally I'd be happy to see any markup language becoming the default."

I don't agree with this. All lightweight humane markup languages are not born equal, some are better others, and Mediawiki's is not in the list of the best ones. Now there seem to be a trend towards Markdown, but it should be improved and then, migrating Wikipedia to this Markdown2 could be a real good thing.

Re: The Parser that Cracked the MediaWiki Code

#33
post #9

I hadn't realized that there were any parsing issues around MediaWiki's markup. 5000 lines of PHP? Eek.

It's worse. The MediaWiki PHP code doesn't implement a proper scanner and parser, it's a bunch of regexes around which the code has grown more or less organically. Silent compensation for mismatched starting and ending tokens abounds, and causes problems for all consumers of the markup, in the same way that lenient HTML parsers have. The difference is that Wikipedia, as the sole channel for editing markup, could have…

Regarding the storage and access of Wikipedia's "world knowledge", DBpedia is a project that scrapes Wikipedia (InfoBoxes and categories) to create a structured, semantic database of knowledge.

https://secure.wikimedia.org/wikipedia/en/wiki/DBpedia

Post reply on HN