Chrome 59 has cross-platform headless support • Hacker News Board

Re: Chrome 59 has cross-platform headless support

#71

Post by seanp2k2 » Thu, Apr 13, 2017, 2:12 AM UTC

PatrickAuld wrote: post #40

bluepnume wrote: post #10

This is fantastic. I'm using a combination of Chrome and PhantomJS for karma testing right now, for https://github.com/paypal/paypal-checkout and https://github.com/krakenjs/xcomponent . There are hundreds of tests opening up hundreds of iframes and popup windows, and sending a lot of cross-window messages, and that ends up being really memory hungry. Chrome deals pretty well with garbage collection, so long as I'm c…

We also did a POC with PhantomJS and found similar issues, as well as generally flakiness causing too many false negatives. Ended up not using it; I'm hoping this can simplify things are give something more solid to build on.

Same, ended up using ChromeDriver + XVFB on Jenkins. Works but it's still pretty slow.

Re: Chrome 59 has cross-platform headless support

#72

Post by nojvek » Thu, Apr 13, 2017, 2:16 AM UTC

kenshaw wrote: post #30

vvoyer wrote: post #12

Also checkout https://github.com/cyrus-and/chrome-remote-interface for an easy way to fully control those headless instances

If you're using Go, https://github.com/knq/chromedp It's also worth noting that the 57+ series has a nice embedded viewer you can use to view the actual viewport via devtools.

If you want great code completion for chrome's large devtools api with lots of different kinds of object then https://www.npmjs.com/package/chrome-remote-debug-protocol

It dynamically generates Typescript definitions for intelligence and type checking from their protocol.json files.

Vscode chrome debugger uses a fork of it.

I'm the author. /shamelessplug

Re: Chrome 59 has cross-platform headless support

#73

Post by ufmace » Thu, Apr 13, 2017, 2:19 AM UTC

pizza wrote: post #22

masterleep wrote: post #2

Please let this be capable of generating PDFs from HTML from the command line.

Is there an easy way to do the opposite? e.g. read PDFs as HTML (intended to be read through a remote shell), or text?

Easy, not so much, depending on exactly what you want to get out of it. I did a project with this once https://www.idrsolutions.com/jpdf2html5/. Last I checked, they only supported it as a Java library that could generate rather nice looking and complete HTML pages from PDF documents. The output was great, but it was kinda pricey and difficult to work with.

On the opposite side of the complexity level, I have also used this http://www.pdfsharp.com/PDFsharp/ to extract bits of text from PDFs. It's free, but you only get access to the raw PDF text with formatting codes. It works fine if you just want to grab a short string, but you got your work cut out for you if you want to do anything more sophisticated.

Re: Chrome 59 has cross-platform headless support

#74

Post by guiambros » Thu, Apr 13, 2017, 2:32 AM UTC

MichaelApproved wrote: post #44

A lot of phantom js talk here makes me want to recommend http://ghostinspector.com It's a phantom js (and other headless browser) web service. Using the site, you can quickly create different tests, scheduled tests, chained tests, keep screen shots, create videos of multi step tests, and have historical information of it all. Can't say enough good things about the site. Edit: also there's a great chrome extension tha…

+1 to GhostInspector; I used them at a previous company a few years ago, and it was very useful.

They were just starting, but service was rather reliable, and their tech support was excellent (maybe because we were early customers). We used to run a bunch of automated tests for monitoring and compliance, archiving hourly screenshots over different builds for later comparison.

Re: Chrome 59 has cross-platform headless support

#75

Post by adam77 » Thu, Apr 13, 2017, 2:33 AM UTC

erikig wrote: post #54

vvoyer wrote: post #9

Earlier quoted context omitted.

Yes: https://chromedevtools.github.io/debugger-protocol-viewer/to...

https://wkhtmltopdf.org/ You can download the two tools wkhtmltopdf and wkhtmltoimage which use WebKit to generate pdfs/images.

Sadly wkhtmltopdf's version of Webkit is ancient :( No flexbox, no es6, etc...

Re: Chrome 59 has cross-platform headless support

#76

Post by chrissnell » Thu, Apr 13, 2017, 2:36 AM UTC

nreece wrote: post #68

I've been testing Chrome headless extensively for the past few months, and while it's a good step, but it's not stable for high-volume or even diverse set of webpages. Memory usage is pretty high, lot of heavy webpages result in crashes/hangs, there are many inconsistencies between features available in full version and headless, their debugging protocol has different APIs that work on headless/non-headless in Linux…

Yes, this exactly. I wrote Crabby [1] a few months back to schedule automated page testing using Chrome and webdriver but doing anything automated with Chrome is really atrocious. You can't expect to load more than one page every 10-15 seconds on the average 8GB instance and it occasionally crashes or otherwise stops working completely.

I ended up writing a simple check using Go's net/http library to do basic performance profiling but it doesn't measure DOM loading like the Chrome checks do. Such a bummer

What I really want is an easy, cross-platform way to collect the network timings for each object like you get in Chrome's dev tools network waterfall graphs.

[1] https://github.com/chrissnell/crabby

Re: Chrome 59 has cross-platform headless support

#77

Post by eppsilon » Thu, Apr 13, 2017, 2:42 AM UTC

chews wrote: post #5

zwerdlds wrote: post #3

Earlier quoted context omitted.

FWIW, you can do that now, using phantomjs (which is chrome) http://phantomjs.org/screen-capture.html

specifically it's qtwebkit and an old version of it.

They recently released a beta version based on a newer version of QtWebKit:

https://bitbucket.org/ariya/phantomjs/downloads/

Re: Chrome 59 has cross-platform headless support

#78

Post by chrissnell » Thu, Apr 13, 2017, 2:42 AM UTC

kenshaw wrote: post #30

vvoyer wrote: post #12

Also checkout https://github.com/cyrus-and/chrome-remote-interface for an easy way to fully control those headless instances

If you're using Go, https://github.com/knq/chromedp It's also worth noting that the 57+ series has a nice embedded viewer you can use to view the actual viewport via devtools.

WOW. That's fantastic. As I noted in a comment above, I wrote a tool called crabby that uses Selenium and Chrome headless to do automated page testing and to report results back to metrics engines like Graphite, Datadog, Prometheus, and Riemann. The biggest problem I have is the unreliability of chromedriver and the extreme resource consumption of Chrome + Selenium. It's really too much for your average public cloud instance if you want to test any more than, say, one or two pages a minute.

Do you know if chromedp can access any of the timing measurements?

Re: Chrome 59 has cross-platform headless support

#79

Post by dvallet » Thu, Apr 13, 2017, 2:49 AM UTC

notatoad wrote: post #61

llimllib wrote: post #57

Earlier quoted context omitted.

same for me. Wonder if we're doing something wrong?

Has it actually landed in the builds yet, or just been marked as targeting 59 in the platform docs?

We are targeting 59. A fix for Mac OS X is on its way: https://codereview.chromium.org/2811633002

Re: Chrome 59 has cross-platform headless support

#80

Post by bluepnume » Thu, Apr 13, 2017, 2:59 AM UTC

Rumudiez wrote: post #34

bluepnume wrote: post #10

This is fantastic. I'm using a combination of Chrome and PhantomJS for karma testing right now, for https://github.com/paypal/paypal-checkout and https://github.com/krakenjs/xcomponent . There are hundreds of tests opening up hundreds of iframes and popup windows, and sending a lot of cross-window messages, and that ends up being really memory hungry. Chrome deals pretty well with garbage collection, so long as I'm c…

What do you do to ensure the resources for closed windows are released?

A combination of:

- Making sure all promises are fulfilled or rejected, so window objects don't get caught indefinitely in closure scope for any .then() or .catch() handler functions.

- Using WeakMaps as much as possible, when we have things that are tied to a particular window, like message listeners or response handlers in post-robot

- Manually clearing up any global references to windows when we destroy an xcomponent instance

Finding the references was the tricky bit. A lot of the effort was finding a leaky test-case, running in 100 times in succession, and deleting code until the memory graph was flat -- then figuring out what I'd just deleted that caused the leak.

The problem started manifesting as I added more and more tests -- so now I'm actually checking my tests' memory usage on the fly and failing if they cross a threshold. Hopefully that should avoid getting into this kind of sticky situation ever again.

https://github.com/paypal/paypal-checkout/blob/master/test/t...