Live data from Hacker News

Google Open-Sources Gumbo: C Library for Parsing HTML5

github.com

21–30 of 63 posts

Re: Google Open-Sources Gumbo: C Library for Parsing HTML5

#21
post #13

Are there plans to add bindings for PHP? Would be awesome.

I don't have plans to. It's an open-source library, though, so there's nothing stopping an enterprising programmer familiar with PHP extensions to add some herself. That's what Gumbo was designed for: to serve as a building block for other tools.

any chance to open source DOM too? Thug made a solid DOM with pyV8 but still not 100% compatible as far as i know.

Re: Google Open-Sources Gumbo: C Library for Parsing HTML5

#24
post #22

Um... Incoming Naive Questions. Why do we need yet another HTML5 Parser? What's wrong with Webkit? What's wrong with the new Gecko2 HTML5 parser? And what license is it?

License is Apache 2: https://github.com/google/gumbo-parser/blob/master/COPYING

Re: Google Open-Sources Gumbo: C Library for Parsing HTML5

#25
post #22

Um... Incoming Naive Questions. Why do we need yet another HTML5 Parser? What's wrong with Webkit? What's wrong with the new Gecko2 HTML5 parser? And what license is it?

Because WebKit and Gecko are not only parsers for starters. They are much more complex layout engines. Which is a whole other can of worms.

Re: Google Open-Sources Gumbo: C Library for Parsing HTML5

#26

Earlier quoted context omitted.

Although not anyone can crawl 2+ billion pages AND fix all the crazy edge cases without going insane :)

Who says I didn't go insane? ;-) The crazy edge cases were...challenging. The source code to the parser is very assert-heavy, so if there's anything that's amiss, it tends to blow up with an assertion failure. I'd run the MapReduce and it would blow up a few hundred times, then MapReduce would stop trying and kill the job. Then when I had a spare moment, I'd look at the assertion failures, pick off the most common on…

I like programming assert-heavy C. As soon as something is out of wack with my mental model the whole thing explodes. I sure as hell don't want to try handling things I don't already understand.

Re: Google Open-Sources Gumbo: C Library for Parsing HTML5

#27
post #5

How does this compare to Hubbub, another C library for parsing HTML5? http://www.netsurf-browser.org/projects/hubbub/

From a cursory glance at the Hubbub source: 1. Hubbub provides a SAX-style (callback-based) API, while Gumbo gives you a DOM-style (tree-based) struct directly. Hubbub is likely faster in this regard, Gumbo is easier to use out-of-the-box. 2. Gumbo is better tested. It's unclear whether Hubbub's 90% test coverage is "90% of the code is tested" or "90% of the tests pass", but Gumbo has 100% code coverage, 100% of html…

Have you seen this before? http://vtd-xml.sourceforge.net/faq.html#How_do_I_get_started...

It's an xml parser that is neither DOM nor SAX. I haven't seen much mention of it before, except as a recommendation for Java devs. There's a C version too. It makes bold claims about performance.

"Comparing with DOM, VTD-XML is significantly faster (up to 10x), more memory-efficient (up to 5x).

Comparing with SAX/PULL, VTD-XML is not only faster, but also is capable of random-access, therefore is easier to use."

Basically by building a DOM style model in SAX fashion.

Re: Google Open-Sources Gumbo: C Library for Parsing HTML5

#28
Node.js bindings would be awesome, especially for writing a headless website parser.

I just forked Gumbo and gave it a shot, but my limited experience of v8 didn't get me very far. I was able to create and build a basic plugin, but returning the scope with a Gumbo object was beyond my limited capabilities.

I think as a little side project I may continue to work on this.

Re: Google Open-Sources Gumbo: C Library for Parsing HTML5

#29
post #22

Um... Incoming Naive Questions. Why do we need yet another HTML5 Parser? What's wrong with Webkit? What's wrong with the new Gecko2 HTML5 parser? And what license is it?

Because WebKit and Gecko are not only parsers for starters. They are much more complex layout engines. Which is a whole other can of worms.

The new Gecko parser is based on a Java->C++ translation from this standalone parser: http://about.validator.nu/htmlparser/
Post reply on HN