Show HN: Getting started with Puppeteer and Chrome Headless for Web Scraping • Hacker News Board

Re: Show HN: Getting started with Puppeteer and Chrome Headless for Web Scraping

#11

Post by kasbah » Tue, Aug 29, 2017, 4:47 AM UTC

Seems like most of the parsing is done by JSDOM in this tutorial.

Re: Show HN: Getting started with Puppeteer and Chrome Headless for Web Scraping

#12

Post by andrewguenther » Tue, Aug 29, 2017, 5:00 AM UTC

MrBlue wrote: post #10

Puppeteer is definitely cool but on a recent project I had to revert back to using NightmareJS as I needed to download files.

This is currently being worked on in Headless Chrome. There's been tons of development on the project and they're super open to feature requests.

Re: Show HN: Getting started with Puppeteer and Chrome Headless for Web Scraping

#13

Post by veb » Tue, Aug 29, 2017, 5:38 AM UTC

Ooooh I read this fantastic introduction the other day and wrote this wee HN demo using Cheerio. https://gist.github.com/veb/c1beab69b5eb1b07123e5eaf55b80320

Re: Show HN: Getting started with Puppeteer and Chrome Headless for Web Scraping

#14

Post by twsted » Tue, Aug 29, 2017, 5:40 AM UTC

Two things:

1. Please do not test a web app with Chrome only, we don't want to go back to a world with a single browser

2. > So, until puppeteer supports this, we will rely on jsdom, a package available via npm

JSDOM is not just a package on npm, it's an engineering piece of art

Re: Show HN: Getting started with Puppeteer and Chrome Headless for Web Scraping

#15

Post by emadehsan » Tue, Aug 29, 2017, 6:18 AM UTC

kasbah wrote: post #11

Seems like most of the parsing is done by JSDOM in this tutorial.

Updated to use `page.evaluate`

Re: Show HN: Getting started with Puppeteer and Chrome Headless for Web Scraping

#16

Post by testcross » Tue, Aug 29, 2017, 6:20 AM UTC

Is it possible to launch multiple times const browser = await puppeteer.launch(); in a same nodejs process? I haven't find any information about that

Re: Show HN: Getting started with Puppeteer and Chrome Headless for Web Scraping

#17

Post by ankit84 » Tue, Aug 29, 2017, 6:21 AM UTC

Great tutorial! Also, you look like a Full stack. How's the reception for Hospital Run software you worked on? (https://github.com/HospitalRun/hospitalrun-frontend)

""Somewhat similar is the case with Internet that we traversed today in quest of data.""

Re: Show HN: Getting started with Puppeteer and Chrome Headless for Web Scraping

#18

Post by testcross » Tue, Aug 29, 2017, 6:23 AM UTC

desireco42 wrote: post #9

I tried it out when it was released, it works well and it is decently fast.

What is faster than puppeteer? All the alternatives using electron look slower.

Re: Show HN: Getting started with Puppeteer and Chrome Headless for Web Scraping

#19

Post by aslushnikov » Tue, Aug 29, 2017, 6:38 AM UTC

testcross wrote: post #16

Is it possible to launch multiple times const browser = await puppeteer.launch(); in a same nodejs process? I haven't find any information about that

It is possible. Beware though that each `puppeteer.launch()` will spawn a chromium process.

Re: Show HN: Getting started with Puppeteer and Chrome Headless for Web Scraping

#20

Post by emadehsan » Tue, Aug 29, 2017, 6:39 AM UTC

ankit84 wrote: post #17

Great tutorial! Also, you look like a Full stack. How's the reception for Hospital Run software you worked on? ( https://github.com/HospitalRun/hospitalrun-frontend ) ""Somewhat similar is the case with Internet that we traversed today in quest of data.""

HospitalRun team is great and very welcoming. You can join there Slack channel here: https://hospitalrun.slack.com/ . The project is expected to be undertaken by JS Foundation in near future. And Yeah, I am a Full stack developer.