Live data from Hacker News

Google Open-Sources Gumbo: C Library for Parsing HTML5

github.com

31–40 of 63 posts

Re: Google Open-Sources Gumbo: C Library for Parsing HTML5

#31

Node.js bindings would be awesome, especially for writing a headless website parser. I just forked Gumbo and gave it a shot, but my limited experience of v8 didn't get me very far. I was able to create and build a basic plugin, but returning the scope with a Gumbo object was beyond my limited capabilities. I think as a little side project I may continue to work on this.

Nice idea, give the repo an I'll give it a push =)

Re: Google Open-Sources Gumbo: C Library for Parsing HTML5

#32
post #22

Um... Incoming Naive Questions. Why do we need yet another HTML5 Parser? What's wrong with Webkit? What's wrong with the new Gecko2 HTML5 parser? And what license is it?

A standalone parser written in C is a great asset. Pretty much any language worth mentioning has C bindings, so they are now just a bindings implementation away from having a reasonably fast (the fact that performance was a non-goal notwithstanding), standards compliant HTML parser. This is an improvement over the status-quo where most languages have bindings to lxml which is fast but has made-up error handling and a tendency to deal poorly with quite a lot of content, and some languages have slow, native implementations of the HTML standard parsing algorithm (I wrote much of Python's html5lib so I am aware both that it is slow and that it is non-trivial to speed up).

Compared to Gecko and WebKit, this gives you just the parser, which is significantly simpler than the whole engine and all you want for many applications.

Re: Google Open-Sources Gumbo: C Library for Parsing HTML5

#33

Node.js bindings would be awesome, especially for writing a headless website parser. I just forked Gumbo and gave it a shot, but my limited experience of v8 didn't get me very far. I was able to create and build a basic plugin, but returning the scope with a Gumbo object was beyond my limited capabilities. I think as a little side project I may continue to work on this.

Nice idea, give the repo an I'll give it a push =)

Currently it's just a straight fork - https://github.com/jbrooksuk/gumbo-parser

I'm giving it another crack now.

You can see my first attempt at https://github.com/jbrooksuk/gumbo-parser/commit/d64f78b125f... - unfortunately it's not quite there. I can't figure out how to send back the struct created by Gumbo.

Re: Google Open-Sources Gumbo: C Library for Parsing HTML5

#34
I'm curious as to what was the motivation behind this project at Google. It seems to me that the only benefit of writing something like this in pure C would be performance gains over existing parsers, but it specifically says in the README that parsing performance was not one of the goals.

Re: Google Open-Sources Gumbo: C Library for Parsing HTML5

#35

Node.js bindings would be awesome, especially for writing a headless website parser. I just forked Gumbo and gave it a shot, but my limited experience of v8 didn't get me very far. I was able to create and build a basic plugin, but returning the scope with a Gumbo object was beyond my limited capabilities. I think as a little side project I may continue to work on this.

Nice idea, give the repo an I'll give it a push =)

[deleted]

Re: Google Open-Sources Gumbo: C Library for Parsing HTML5

#37

Nice to see they've written the parser from scratch rather than some BISON and FLEX construct that generates impenetrable code.

Just like with any generated code, you are not supposed to read the code bison and flex generate (except maybe if you are a flex/bison developer). 'Impenetrable' is neither an advantage nor a disadvantage of generated code.

Do you often read the output of some library macros that the C pre-compiler generates?

Re: Google Open-Sources Gumbo: C Library for Parsing HTML5

#38

Earlier quoted context omitted.

I'd be interested in reading about your learning when moving from a UI -> Algo heavy engineer :)

It's a long story, and it's also not complete yet (I'm actually doing very UI heavy work right now as a tech lead). It's also not really correct to say it started with UI - I was big into programming language theory in college, even implementing a bunch of toy interpreters/compilers, one of which even got some measure of fame on the Internet. The 5 second overview is really that it's the same as getting good at any n…

This is the most inspiring thing I've read in a long time. There's a lot of chatter on HN about how people became an "expert" in this or that, but for some reason, the way you phrased it really resonated with me. And to see the end result -- holy crap. HTML is complicated.

Re: Google Open-Sources Gumbo: C Library for Parsing HTML5

#39

Earlier quoted context omitted.

Who says I didn't go insane? ;-) The crazy edge cases were...challenging. The source code to the parser is very assert-heavy, so if there's anything that's amiss, it tends to blow up with an assertion failure. I'd run the MapReduce and it would blow up a few hundred times, then MapReduce would stop trying and kill the job. Then when I had a spare moment, I'd look at the assertion failures, pick off the most common on…

I like programming assert-heavy C. As soon as something is out of wack with my mental model the whole thing explodes. I sure as hell don't want to try handling things I don't already understand.

You can use asserts in an exploratory fashion to document code you don't understand too.

Re: Google Open-Sources Gumbo: C Library for Parsing HTML5

#40
post #30

Can you comment on why this is C and not C++? (Although some examples are in C++)

If you are writing this as a library you're most likely going to end up having to write a C wrapper for most language bindings. In the end it may just be easier to write the whole library in C and then write the specific binding for Python, Ruby, etc.
Post reply on HN