Earlier quoted context omitted.
Chromium is Open Source and embeddable, so my guess is a big part of why their headless variant isn't open source is simply that there isn't really much to it.
From a naive point of view it's easy. But it has to scale and launching a new process for every request works only for a pet project. So a crawler based on a headless browser that consumes little memory and runs for weeks is a major achivement.
Re: Does Google crawl dynamic content?
#61Why doesn't that scale? You might be underestimating Google's resources.