Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: Run Puppeteer on AWS Lambda

github.com

Post reply on HN
Print view View original

21–25 of 25 posts

Re: Show HN: Run Puppeteer on AWS Lambda

#21

Post by alixaxel »

tnolet wrote: post #17

I run a ton of Puppeteer jobs (300k in the last month), currently on EC2 and Digital Ocean VM's, mostly due to the subtle difficulties of running Puppeteer on Lambda. Will certainly have a look at this project and contribute where possible. My main concerns are not so much cold start time, as for my use case this is not really a huge issue, but mainly the performance of Chrome on AWS Lambda boxes. The rendering, navi…

I also run hundreds of thousands of puppeteer sessions every month, all on Lambda and so far I'm pretty happy with it, from scalability itself to session performance.

Granted, there are some issues with rendering (fonts, emojis and whatnot) but meanwhile there are solutions available that could be explored.

Feel free to try it out and share your specific challenges on GitHub, I'll do my best to come up with solutions for them.

Re: Show HN: Run Puppeteer on AWS Lambda

#22

Post by thesandlord »

alixaxel wrote: post #20
thesandlord wrote: post #19

Earlier quoted context omitted.

Google App Engine and Google Cloud Functions got native support for Puppeteer a few months ago as well. Let me know what you think if you try it out. https://news.ycombinator.com/item?id=17795626 (I work for Google Cloud)

The performance of puppeteer is super bad on CGF (you can read more about it here https://github.com/GoogleChrome/puppeteer/issues/3120 ). It would actually be great to have someone really improve this situation instead of dismissing it as a weird IO problem.

Did some research internally, this is being tracked but still no root cause AFAIK :(

Re: Show HN: Run Puppeteer on AWS Lambda

#23

Post by russian_bot »

alixaxel wrote: post #21
tnolet wrote: post #17

I run a ton of Puppeteer jobs (300k in the last month), currently on EC2 and Digital Ocean VM's, mostly due to the subtle difficulties of running Puppeteer on Lambda. Will certainly have a look at this project and contribute where possible. My main concerns are not so much cold start time, as for my use case this is not really a huge issue, but mainly the performance of Chrome on AWS Lambda boxes. The rendering, navi…

I also run hundreds of thousands of puppeteer sessions every month, all on Lambda and so far I'm pretty happy with it, from scalability itself to session performance. Granted, there are some issues with rendering (fonts, emojis and whatnot) but meanwhile there are solutions available that could be explored. Feel free to try it out and share your specific challenges on GitHub, I'll do my best to come up with solutions…

hi,

out of curiosity, what is it that you do that demands so many sessions? Just webscraping?

Re: Show HN: Run Puppeteer on AWS Lambda

#24

Post by tnolet »

thesandlord wrote: post #19
tnolet wrote: post #17

I run a ton of Puppeteer jobs (300k in the last month), currently on EC2 and Digital Ocean VM's, mostly due to the subtle difficulties of running Puppeteer on Lambda. Will certainly have a look at this project and contribute where possible. My main concerns are not so much cold start time, as for my use case this is not really a huge issue, but mainly the performance of Chrome on AWS Lambda boxes. The rendering, navi…

Google App Engine and Google Cloud Functions got native support for Puppeteer a few months ago as well. Let me know what you think if you try it out. https://news.ycombinator.com/item?id=17795626 (I work for Google Cloud)

Would love to use GCF but the performance is terrible (as mentioned) and I need more geographic locations than GCF offers.
Post reply on HN