Live data from Hacker News

Prototype: Puppeteer for Firefox

github.com

61–70 of 133 posts

Re: Prototype: Puppeteer for Firefox

#61
For what I see, puppeteer is very much like testcafe (able to run with headless/full Firefox, Chrome, Edge ..., but testcafe doesn't need a custom-built Firefox). TestCafe can run javascript code on the page and return the data back, that means it can be used to get data in POST requests. It can take screenshots too. If anybody is familiar with both tools, could you please what this tool can do but the other cannot?

Re: Prototype: Puppeteer for Firefox

#62
post #55
post #15

For automated testing, why Puppeteer instead of Selenium?

Selenium feels old and outdated, it's always a hustle to install it/get it running and I have to use WebdriverIO with it in Node. I tried Puppeteer a few days ago, I just did `npm i puppeteer` and it worked as expected.

How would you describe the npm install process?

Re: Prototype: Puppeteer for Firefox

#63
post #61

For what I see, puppeteer is very much like testcafe (able to run with headless/full Firefox, Chrome, Edge ..., but testcafe doesn't need a custom-built Firefox). TestCafe can run javascript code on the page and return the data back, that means it can be used to get data in POST requests. It can take screenshots too. If anybody is familiar with both tools, could you please what this tool can do but the other cannot?

I have never heard of TestCafe, but I've used Puppeteer for a variety of projects with Chrome. My latest one was using it to convert HTML files into PDFs en masse.

Re: Prototype: Puppeteer for Firefox

#64
post #57
post #46

Earlier quoted context omitted.

It's probably because in the meantime dom nodes has been replaced by new ones. Selenium has it's quirks but when you understand how it works and it's limitations and user centric philosophy there is nothing that can stop you from writing rock solid stable tests.

Sorry I didn't defined the problems clearly and thank your for your help. I am using selenium using python for web automation, not testing. The latter problem was by and large solved by subclassing WebDriverWait by adding common Exceptions related to nodes staleness. The former problem was tackled by writing a for loop detecting changes of the context to know if WebElement.click() fails silently. The thing is I have…

Selenium for sure doesn't fit in your use case then :). I pointed "user-centric approach" because selenium focuses on allowing only things normal user could do in normal user way which is limiting for automation but it's a blessing if you want to look as close to the real user as you can. I worked on solving google ReCaptcha v2/v3 at bigger scale and it's been a solid advantage.

> And javascript code in a string of python can not be linted, which increases debugging workload.

If you use PyCharm or any other JetBrains IDE you can use `Language Injections` [0]

> I am wondering if using puppeteer can ease pains mentioned above.

I'm not sure about puppeteer but Chrome DevTools Protocol [1] (I think this is what puppeteer use under the hood, but I'm not sure) may be interesting for you because it gives low level access to browser mechanisms like injecting code before page load, request interception or separate browser contexts between separate tabs.

[0] https://www.jetbrains.com/help/pycharm/using-language-inject...

[1] https://chromedevtools.github.io/devtools-protocol/

Re: Prototype: Puppeteer for Firefox

#65
post #61

For what I see, puppeteer is very much like testcafe (able to run with headless/full Firefox, Chrome, Edge ..., but testcafe doesn't need a custom-built Firefox). TestCafe can run javascript code on the page and return the data back, that means it can be used to get data in POST requests. It can take screenshots too. If anybody is familiar with both tools, could you please what this tool can do but the other cannot?

I have never heard of TestCafe, but I've used Puppeteer for a variety of projects with Chrome. My latest one was using it to convert HTML files into PDFs en masse.

Is there an advantage to that approach over the more direct wkhtmltopdf?

Re: Prototype: Puppeteer for Firefox

#66
post #50

This is nice work. Does anyone know if it's possible to enable reader view "programatically", though?

Sure, just go to `about:reader?url=whatever` Or did you mean something else?

This is exactly what I'm after. Thanks a lot! :)

Re: Prototype: Puppeteer for Firefox

#67
Puppeteer is really really good. I just discovered it and used it for something and it’s a blast to work with.

I’m seriously thinking there is a cloud api play in the near future — a puppeteer Page is a pretty awesome container format ... a Puppeteer Page or BrowserContext per request would provide an awesome cloud-function-like programming model I think...

Re: Prototype: Puppeteer for Firefox

#68
post #37

Earlier quoted context omitted.

If it really E2E from user perspective you shouldn't doing any magic and manipulate POST data.

You're not technically wrong, but unit/integration/E2E are not hard categories for tests, they're just points along a continuum. Sometimes I want to directly control a browser to run a test or examine a UI component, but I still want to mock parts of my application. This is one of the biggest weaknesses with Selenium -- it takes a very narrow view of what testing is, so it becomes much less useful as soon as you're t…

> Sometimes I want to directly control a browser to run a test or examine a UI component, but I still want to mock parts of my application.

If you're building a SPA, you can mock out the backend and control the backend mock behaviour in your Selenium test.

Re: Prototype: Puppeteer for Firefox

#69
post #15

For automated testing, why Puppeteer instead of Selenium?

With WebDriver becoming a standard it's sad the browser vendors didn't work together to improve that. Instead Google made their own thing and now the world has to follow

Agreed. Any idea why Google didn't work with WebDriver and decided to spin-off yet another homegrown tech?

My conspiracy-laden mind veers towards the obvious but I'd prefer a better explanation.

Re: Prototype: Puppeteer for Firefox

#70
post #43
post #41

Earlier quoted context omitted.

I just used Chrome headless to render several thousand pdfs last week and I saw no issues in version 73

Did you have a reason to not use PDFium directly?

Likely part of a web scrapping project that may require navigation or other browser things to get to. Or they are trying to avoid detection with a more real browser (you have to change some stuff first).
Post reply on HN