Live data from Hacker News

Show HN: Chromeless – Headless Chrome Automation on AWS Lambda

github.com

51–60 of 92 posts

Re: Show HN: Chromeless – Headless Chrome Automation on AWS Lambda

#51
post #45

This is pretty nifty! I've been keeping an eye on Phantomium for a while, I wonder what's come out of it.

Looks like Google releasing headless Chrome is really shaking up the test automation domain. This is a really cool project :) The other day I saw another interesting and refreshing implementation using GraphQL - https://github.com/joelgriffith/navalia (not affiliated with the project, just got to know from a PhantomJS issue).

Re: Show HN: Chromeless – Headless Chrome Automation on AWS Lambda

#52

I've been super excited about Chrome headless but haven't had a chance to dig into using it yet. The api here looks amazing for getting started without getting lost in the weeds. It'd be fairly trivial to hook this up to a Slackbot and to get on-demand screenshots of various pages on my websites, etc.

A month ago I was trying to migrate some tests from phantomjs to chrome headless. The big blockers I ran into were that chromedriver automation has certain functionality that only works with a special chrome extension that it installs- but headless chrome doesn't support extensions.

There's work being done to fix this, but it's still in progress, last I checked. Until then, resizing windows, taking screenshots and a handful of other things simply don't work.

Re: Show HN: Chromeless – Headless Chrome Automation on AWS Lambda

#53
shamless plug: i've also written a high level api on top of the chrome remote debugger chrominator [1]

similar idea. chrominator use promises instead of a fluent api. it also follows the selenium w3c spec where possible. it does cool stuff with evaluate and evaluateAsync where it resolves the remote object to something usable.

to be fair there are a few other projects i know about that wrap chrome remote debugger with a high level api:

* autogcd [2]

* ghostjs [3]

[1] https://github.com/jesg/chrominator

[2] https://github.com/wirepair/autogcd

[3] https://github.com/KevinGrandon/ghostjs

Re: Show HN: Chromeless – Headless Chrome Automation on AWS Lambda

#55
post #50

I've got the impression that lots of sites block AWS IP addresses. I wonder if this would hamper the practical use of this on Lambda. I'm doing something similar, and this concern was one motivation for running in our datacentre vs EC2. Does anyone have concrete info on rates of bots blocked from AWS IPs?

I assume the number one use of this would be test automation for one's own sites so blocking would not be an issue. What are sites' motivations for blocking AWS IPs? I bet there are some reasons I would agree with even though the somewhat crude method of blocking ip range would have some unintended consequences (e.g. blocking people running a personal VPN).

>What are sites' motivations for blocking AWS IPs?

I block AWS. So many crawlers up to so much nonsense! I don't block by IP, but by hostname.

  $block='.amazonaws.com';
  $ua = @$_SERVER['HTTP_USER_AGENT'];
  
  if (stripos($rh,$block)!==false &&
  	stripos($ua,'Silk')===false &&
  	stripos($ua,'Safari')===false){
  
    	$block_visitor=true;
  	$message="Blocked Host:Amazon Web Services";
  }

Re: Show HN: Chromeless – Headless Chrome Automation on AWS Lambda

#59
post #53

shamless plug: i've also written a high level api on top of the chrome remote debugger chrominator [1] similar idea. chrominator use promises instead of a fluent api. it also follows the selenium w3c spec where possible. it does cool stuff with evaluate and evaluateAsync where it resolves the remote object to something usable. to be fair there are a few other projects i know about that wrap chrome remote debugger wit…

[1] Chrominator looks interesting. jesg/gowan is the maintainer for PhantomJS GhostDriver, so the API looks extensive. Another project that I heard recently is Navalia [4], which uses GraphQL type of queries to run headless Chrome. Also worth noting is Chromy [5] by a Japanese developer; it's one of the earliest to market (released in early May). BackstopJS maintainer also looks to be in active development using Chromy.

[4] https://github.com/joelgriffith/navalia

[5] https://github.com/OnetapInc/chromy

Re: Show HN: Chromeless – Headless Chrome Automation on AWS Lambda

#60

I've got the impression that lots of sites block AWS IP addresses. I wonder if this would hamper the practical use of this on Lambda. I'm doing something similar, and this concern was one motivation for running in our datacentre vs EC2. Does anyone have concrete info on rates of bots blocked from AWS IPs?

Fairly simple (if this is like the other programmatic headless browsers) to make a request through a proxy.
Post reply on HN