Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Puppeteer: Headless Chrome Node API

github.com

Post reply on HN
Print view View original

81–90 of 105 posts

Re: Puppeteer: Headless Chrome Node API

#81

Post by rbobby »

DonHopkins wrote: post #70
fake-name wrote: post #54

Earlier quoted context omitted.

There is complexity here you may not be seeing, namely if you are on a platform without X, it will not work. Normal chromium requires a bunch of X libraries be present. It doesn't use them, but for things like headless testing, it's a massive pain, since the apt-get (or equivalent) is generally many hundreds of megabytes.

X: The First Fully Modular Software Disaster http://www.art.net/~hopkins/Don/unix-haters/x-windows/disast...

> If you're using the Motif self-abuse kit

hehehehehe

Re: Puppeteer: Headless Chrome Node API

#82

Post by Akujin »

paulirish wrote: post #5
Akujin wrote: post #2

I think simple-headless-chrome is more far along https://github.com/LucianoGanga/simple-headless-chrome

We're fans of LucianoGanga's project (and of Chromeless, Doffy, Chrominator, Chromy, Navalia). I can tell you from personal experience that dealing with the raw DevTools Protocol isn't ideal for a developer writing an automation script, so it's clear there's demand for libraries with this higher-level API. Would love to know if there's a feature parity concern you have or what you'd like to see from puppeteer (or any…

Your unicorn would be lovely though :D

Re: Puppeteer: Headless Chrome Node API

#83

Post by kensoh »

fake-name wrote: post #58
bauerd wrote: post #17

Earlier quoted context omitted.

Yes, I wholeheartedly agree, it was a stupid decision by the Selenium devs not to make request headers etc. accessible. But why throw all the standardisation efforts overboard?

I think this is more "selenium is actively antagonistic to it's major use-case", then trying to throw everything away. There have been multiple attempts to convince the selenium people to revisit their decision W.R.T. headers, and they're completely unwilling. Given that the selenium leadership is apparently uninterested in improvements, and it's many limitations, trying to improve there is more effort then it's wort…

I didn't follow that development. Can you share why Selenium maintainers chose not to implement headers? Is it that they want to restrict the tool to simulate what a normal user can do with a browser and not hacks such as overriding headers? Thanks in advance!

Re: Puppeteer: Headless Chrome Node API

#84

Post by kensoh »

adamb_ wrote: post #79
pimlottc wrote: post #76

Earlier quoted context omitted.

I'm not super familiar with this space, can you explain further why everyone needs to switch from PhantomJS?

Core developer stepped down: https://news.ycombinator.com/item?id=14105489

There was still some commits in early July but haven't seen new updates till date. Maintaining a browser project is mammoth task. Even for an established startup such as Segment you see that NightmareJS has lots of issues unreplied for months. I guess it's primarily because of a potentially large user base and all kinds of edge cases requirements from different users. And add on to that, suppose to work on all OSes. Nightmare.

Re: Puppeteer: Headless Chrome Node API

#85

Post by bluepnume »

I'm really loving headless chrome so far. I have around 650 tests which are mostly dealing with iframes and popup windows, and they run flawlessly. The first release seemed to have a memory leak which wasn't present in non-headless chrome, but that seems to have been fixed in version 60.

Even sped things up a little versus phantom.

Re: Puppeteer: Headless Chrome Node API

#86

Post by kensoh »

ghetzel wrote: post #62
samwillis wrote: post #26

Has anyone found something similar but for Python? The few I found all seemed to be abandoned or too limited in capability.

I've been working on something similar (Headless Chrome via DevTools protocol) called Webfriend. It is a Python wrapper to the DevTools protocol, as well as a simplified imperative scripting environment which is specifically built for ease of use by people with a technical-but-not-programming background (lovingly called Friendscript). It's by no means done, but it is functional and I'm hoping to see the project grow…

Thanks ghetzel for sharing, nice work! I saw that you've made your own scripting language to make it user-friendly :)

Just went through your Friendscript syntax, the amount of work going into defining that language is impressive....

Re: Puppeteer: Headless Chrome Node API

#87

Post by kensoh »

kensoh wrote: post #41
bdcravens wrote: post #25

Has headless Chrome enabled file downloads yet? (I don't mean navigating to a known url and saving content, but when a site pops open a save dialog)

This issue is being tracked here and moving along fine - https://bugs.chromium.org/p/chromium/issues/detail?id=696481 Default behavior by design is to block automated headless downloads for security reasons. But above issue tries to address this important use case.

This's the Puppeteer issue tracking downloads - https://github.com/GoogleChrome/puppeteer/issues/299

Re: Puppeteer: Headless Chrome Node API

#88

Post by kensoh »

bluepnume wrote: post #85

I'm really loving headless chrome so far. I have around 650 tests which are mostly dealing with iframes and popup windows, and they run flawlessly. The first release seemed to have a memory leak which wasn't present in non-headless chrome, but that seems to have been fixed in version 60. Even sped things up a little versus phantom.

Hi bluepnume, if you are ok, can you share how you manage iframes and popups? I'm using DevTools protocol through websocket directly to communicate with Chrome. Have to do a lot of context handling (frames) and using Target.sendMessageToTarget (popup windows) to deal with them. These 2 features seem to be the harder to handle parts of the interaction layer when doing automation with DevTools Protocol. Thanks in advance!

Re: Puppeteer: Headless Chrome Node API

#89

Post by daveFNbuck »

gervase wrote: post #22
bauerd wrote: post #9

So first there was Selenium's JSON Wire protocol, then came the W3C WebDriver spec and now we're back to browser-specific implementations? As someone who's tried/is trying to automate Firefox/Chrome/Safari/IE in a consistent fashion, my only question is: WHY?

Based on a quick read of the API, my interpretation is that this is not targeting people who are trying to automate every browser, but those who need to automate any browser. In that context, it's dead-simple to use, and someone with very little experience should be able to get a working prototype in under 5 minutes. For my use case, it's closer to "wget/curl with JS processing" than "automating a user's browsing exp…

If you just need wget/curl with js, you can actually do that with the chrome CLI now. Just run your chrome binary with the arguments --headless --disable-gpu --dump-dom

Re: Puppeteer: Headless Chrome Node API

#90

Post by applejuice0 »

equasar wrote: post #67

How can this work on Cloud Functions/Amazon Lambda/Azure Functions without installing the dependencies each time it has to run? Haven't done anything before with those serverless approaches.

AWS Lambda presents a base image ready to be provisioned as multiple instances.

If you are familiar with Docker you can think about Lambda image as a small Docker image, while real work will be done in instances (Docker containers) created from this image.

Usual scenario is to provide ready to go image (with all source code, npm packages being installed, with Chrome Headless plugin, etc). Then AWS/Azure will run VM instances based on the image for almost every function request. Most of the time spinning such lite VMs takes no more than a couple of seconds.

Post reply on HN