Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Parsoid in PHP, or There and Back Again

phabricator.wikimedia.org

Post reply on HN
Print view View original

61–70 of 75 posts

Re: Parsoid in PHP, or There and Back Again

#61

Post by trynewideas »

dingdingdang wrote: post #46
trynewideas wrote: post #5

Reintegrating the parser into Mediawiki's PHP core goes beyond performance. Many prominent MW features -- particularly the visual editor, translation functions, and mobile endpoints -- depend heavily on Parsoid/JS, which required running it as a Node.js microservice, something not all smaller (or especially shared-hosting wikis) could manage for quite some time. Bringing Parsoid closer to core makes it easier for non…

Performance gains of "Parsoid/PHP ... roughly twice as fast on most requests as the original version" are bit more than a casual bonus and into territory of pretty dang awesome in my book.

I totally agree, but on wikis of a smaller scale it's not a difference of performance, it's potentially being able to use some of these MW components for the first time. It could be 10x faster than Parsoid/JS and that's irrelevant compared to being _possible_ on a service that won't let you run Parsoid/JS.

Re: Parsoid in PHP, or There and Back Again

#62

Post by Piskvorrr »

lolphp111 wrote: post #60

Why? The editor needs a frontend in javascript anyways, so why mot handle this all in real time on the client? Now they rewrote in PHP, thats probably one of the worst languages out there, and why not rewrite in something compiled if speed was the main reason for a rewrite? For me PHP sits in the middle as a poor language, and still slow compared to any compiled languages. Also i would want to see some wasm vs php be…

Lots of assumptions from you. "PHP bad, PHP slow, everyone has fast client." Did you assume it was 2010?

Re: Parsoid in PHP, or There and Back Again

#63

Post by cscottnet »

tjpnz wrote: post #37

>Parsoid/JS had very few unit tests focused on specific subsections of code. With only integration tests we would find it difficult to test anything but a complete and finished port. I found this a little frightening given Parsoid/JS is handling user input.

There are thousands of integration tests. The "correct" output of the parser is well-known for a given input, and those test cases have been accumulating for over a decade. But the internal structure of the parser is much more fluid, and so it wasn't (historically) thought worthwhile to try to write tests against that shifting target.

Re: Parsoid in PHP, or There and Back Again

#64

Post by cscottnet »

znpy wrote: post #29

Last time i tried getting mediawiki up and running as a personal wiki i found out that getting parsoid working was quite a mess. hopefully now it will be easier to get a fully fledged wikimedia installation, together with a visual editor.

Yes. We're not there yet, but that's the goal!

Re: Parsoid in PHP, or There and Back Again

#65

Post by cscottnet »

harryf wrote: post #20

Years ago I wrote the re-wrote wiki parser for Dokuwiki (which is used at https://wiki.php.net/ among other places). Originally the parser was scanning a wiki page multiple times using various regular expressions. I used a stack machine as a way to manage the regular expressions, which resulted in being able to parse a page in a single pass - it's documented here - https://www.dokuwiki.org/devel:parser A nice (unexpe…

It appears that you were probably describing the lexer pass in your description of docuwiki. Indeed tokenization is a very hard problem for wikitext. We use a pegjs grammar for it, but it contains less of lookahead/special conditions/novel extensions, etc. It's hard. Wikitext is messy precisely because it was intentionally designed to be easy and forgiving to write.

Seems like we've learned many of the same lessons building our parsers. Markup parsers do seem to be a unique thing, not really like parsing either programming languages or natural languages. If we every meet I'm sure we could happily share a beverage of your choice trading stories.

Re: Parsoid in PHP, or There and Back Again

#66

Post by lovasoa »

cscottnet wrote: post #58

I'm on the team. Part 2 of this post series should have lots of interesting technical details for y'all; be patient, I'm still writing it. But to whet your appetite: we used https://github.com/cscott/js2php to generate a "crappy first draft" of the PHP code for our JS source. Not going for correctness, instead trying to match code style and syntax changes so that we could more easily review git diffs from the crappy…

One question: did you investigate why the PHP version was so much faster than the JS one ? Do you think the performance gains of the PHP versions could be achieved in JS, or do you use any special feature of the PHP interpreter ?

Re: Parsoid in PHP, or There and Back Again

#67

Post by lolphp111 »

Piskvorrr wrote: post #62
lolphp111 wrote: post #60

Why? The editor needs a frontend in javascript anyways, so why mot handle this all in real time on the client? Now they rewrote in PHP, thats probably one of the worst languages out there, and why not rewrite in something compiled if speed was the main reason for a rewrite? For me PHP sits in the middle as a poor language, and still slow compared to any compiled languages. Also i would want to see some wasm vs php be…

Lots of assumptions from you. "PHP bad, PHP slow, everyone has fast client." Did you assume it was 2010?

Well, actually i made zero assumptions. Theres tonnes of markdown editors that work in the browser in real time. I assume the wiki syntax is not that much "heavier" to work with, if it is its another lol for the wiki team.

And by PHP i mean if this rewrite is done partly because of speed, why not build the parser in a compiled language? Its just silly that you have to work with PHP as its one of the worst languages out there in terms of dx and features.

I assume its 2020 where clients are "fast enough" to handle a syntax transformation like markdown -> HTML

Re: Parsoid in PHP, or There and Back Again

#68

Post by Piskvorrr »

lolphp111 wrote: post #67
Piskvorrr wrote: post #62

Earlier quoted context omitted.

Lots of assumptions from you. "PHP bad, PHP slow, everyone has fast client." Did you assume it was 2010?

Well, actually i made zero assumptions. Theres tonnes of markdown editors that work in the browser in real time. I assume the wiki syntax is not that much "heavier" to work with, if it is its another lol for the wiki team. And by PHP i mean if this rewrite is done partly because of speed, why not build the parser in a compiled language? Its just silly that you have to work with PHP as its one of the worst languages o…

"For me PHP sits in the middle as a poor language, and still slow compared to any compiled languages." No assumptions whatsoever.

2020: mobile clients. Yes, they're where the rest of the web was in 2005. Yes, they're ubiquitous.

Re: Parsoid in PHP, or There and Back Again

#69

Post by lolphp111 »

Piskvorrr wrote: post #68
lolphp111 wrote: post #67

Earlier quoted context omitted.

Well, actually i made zero assumptions. Theres tonnes of markdown editors that work in the browser in real time. I assume the wiki syntax is not that much "heavier" to work with, if it is its another lol for the wiki team. And by PHP i mean if this rewrite is done partly because of speed, why not build the parser in a compiled language? Its just silly that you have to work with PHP as its one of the worst languages o…

"For me PHP sits in the middle as a poor language, and still slow compared to any compiled languages." No assumptions whatsoever. 2020: mobile clients. Yes, they're where the rest of the web was in 2005. Yes, they're ubiquitous.

I cant believe users on legacy phones (phones from say 2010-ish era) would be writing new content on wikipedia actively. I dont care if you have the newest iphone, you still dont write anything, you just read. For those 0.00001% of users who actually create new content they most def are not using a phone to do this.

> "For me PHP sits in the middle as a poor language, and still slow compared to any compiled languages." No assumptions whatsoever.

Its not an assumption. PHP is slower than a compiled langauge. simple and easy. Need speed? Dont do it in a compiled language. Period.

Re: Parsoid in PHP, or There and Back Again

#70

Post by subbu_ss »

lovasoa wrote: post #66
cscottnet wrote: post #58

I'm on the team. Part 2 of this post series should have lots of interesting technical details for y'all; be patient, I'm still writing it. But to whet your appetite: we used https://github.com/cscott/js2php to generate a "crappy first draft" of the PHP code for our JS source. Not going for correctness, instead trying to match code style and syntax changes so that we could more easily review git diffs from the crappy…

One question: did you investigate why the PHP version was so much faster than the JS one ? Do you think the performance gains of the PHP versions could be achieved in JS, or do you use any special feature of the PHP interpreter ?

No, we haven't investigated it yet since we haven't had the time to do it. But, we've filed a task for maybe someone to look at it ( https://phabricator.wikimedia.org/T241968 ), but our hunch is that it would be be incorrect to conclude that PHP is faster from JS.
Post reply on HN