Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Benchmark: Puppeteer vs. Selenium vs. Playwright vs. WebDriverIO

blog.checklyhq.com

Post reply on HN
Print view View original

21–30 of 31 posts

Re: Benchmark: Puppeteer vs. Selenium vs. Playwright vs. WebDriverIO

#21

Post by diroussel »

bdcravens wrote: post #6

While interesting, performance is a tiny concern here. Most browser automation is going to have a lot of implicit and explicit waits that dwarf the differences in performance between libraries. Moreover, someone may choose Puppeteer over Selenium for functional differences (for example, Selenium is easier to catch and block)

It’s only a “tiny concern” because the results seem so similar. But if one was drastically slower, then it is important.

However I don’t believe this testing methodology shows the true performance impact of the different frameworks. Only once you have a significant number of assertions and page interactions then to we really see the performance impact of the architectural differences.

Re: Benchmark: Puppeteer vs. Selenium vs. Playwright vs. WebDriverIO

#23

Post by cobbzilla »

The simple code examples on the websites of both Puppeteer [1] and Playwright [2] are impressive in their conciseness and power.

It's been a long time since I did browser automation but it was always much uglier than that.

These new browser drivers look (dare I say) downright fun to write stuff in. It's really tempting me to think of some browser automation side project to play with over a weekend...

[1] https://pptr.dev/ [2] https://playwright.dev/

Re: Benchmark: Puppeteer vs. Selenium vs. Playwright vs. WebDriverIO

#24

Post by bdcravens »

diroussel wrote: post #21
bdcravens wrote: post #6

While interesting, performance is a tiny concern here. Most browser automation is going to have a lot of implicit and explicit waits that dwarf the differences in performance between libraries. Moreover, someone may choose Puppeteer over Selenium for functional differences (for example, Selenium is easier to catch and block)

It’s only a “tiny concern” because the results seem so similar. But if one was drastically slower, then it is important. However I don’t believe this testing methodology shows the true performance impact of the different frameworks. Only once you have a significant number of assertions and page interactions then to we really see the performance impact of the architectural differences.

Sure. In my experience (I've done a ton of this kind of work) well designed flow can take at least 30 seconds, up to several minutes depending on what you're automating. A difference of 5-10 seconds tends to not matter in that situation, and you'll be more interested in things like code ergonomics, memory leaks over time, etc.

Re: Benchmark: Puppeteer vs. Selenium vs. Playwright vs. WebDriverIO

#25

Post by repenaxa »

Mattwmaster58 wrote: post #13

The variation between puppeteer and playwright is surprising. I thought playwright was a fork of puppeteer with a little bit of added functionality mixed in.

That's a popular misconception. It is just the shape of the API that makes them look alike, other than that they are fundamentally different at this point. We've put so much effort into Playwright during the last year, probably 3x we put into Puppeteer itself!

Re: Benchmark: Puppeteer vs. Selenium vs. Playwright vs. WebDriverIO

#26

Post by WalterGR »

JackC wrote: post #12

I'm really excited by what Playwright has been up to on the python side recently with the pytest-playwright package. Browser-testing a Django app can be as simple as: def test_homepage(live_server, page): page.goto(live_server.url) assert page.innerText('#splash-quote') == 'our splash-quote' $ pytest -k test_homepage --browser chromium --browser firefox --verbose tests/test_browser.py::test_homepage[chromium] PASSED…

It’s been some time since I’ve used Selenium, but I don’t remember it being any more complicated than that to test arbitrary websites.

What is that code doing that’s special?

Re: Benchmark: Puppeteer vs. Selenium vs. Playwright vs. WebDriverIO

#27

Post by sharmi »

WalterGR wrote: post #26
JackC wrote: post #12

I'm really excited by what Playwright has been up to on the python side recently with the pytest-playwright package. Browser-testing a Django app can be as simple as: def test_homepage(live_server, page): page.goto(live_server.url) assert page.innerText('#splash-quote') == 'our splash-quote' $ pytest -k test_homepage --browser chromium --browser firefox --verbose tests/test_browser.py::test_homepage[chromium] PASSED…

It’s been some time since I’ve used Selenium, but I don’t remember it being any more complicated than that to test arbitrary websites. What is that code doing that’s special?

Not OP. For one thing, Selenium was not super reliable, at least for me, especially when trying to run multiple instances. Do note that I used the official docker swarm method.

When I was exploring that option, Playwright was still windows only. Super excited to see it is available on Linux and Mac.

Re: Benchmark: Puppeteer vs. Selenium vs. Playwright vs. WebDriverIO

#28

Post by arjun27 »

sharmi wrote: post #27
WalterGR wrote: post #26

Earlier quoted context omitted.

It’s been some time since I’ve used Selenium, but I don’t remember it being any more complicated than that to test arbitrary websites. What is that code doing that’s special?

Not OP. For one thing, Selenium was not super reliable, at least for me, especially when trying to run multiple instances. Do note that I used the official docker swarm method. When I was exploring that option, Playwright was still windows only. Super excited to see it is available on Linux and Mac.

Playwright has always supported all platforms: Win, Linux and Mac. Is there something that didn’t work for you?

Re: Benchmark: Puppeteer vs. Selenium vs. Playwright vs. WebDriverIO

#29

Post by sharmi »

arjun27 wrote: post #28
sharmi wrote: post #27

Earlier quoted context omitted.

Not OP. For one thing, Selenium was not super reliable, at least for me, especially when trying to run multiple instances. Do note that I used the official docker swarm method. When I was exploring that option, Playwright was still windows only. Super excited to see it is available on Linux and Mac.

Playwright has always supported all platforms: Win, Linux and Mac. Is there something that didn’t work for you?

Maybe it didn't have python support? Sorry, I may have misremembered. I do remember something missing that I couldn't use it in my existing dev setup. As solo developer, I prefer to keep my dependencies minimal, mostly python.

Re: Benchmark: Puppeteer vs. Selenium vs. Playwright vs. WebDriverIO

#30

Post by samdamsamma »

JackC wrote: post #12

I'm really excited by what Playwright has been up to on the python side recently with the pytest-playwright package. Browser-testing a Django app can be as simple as: def test_homepage(live_server, page): page.goto(live_server.url) assert page.innerText('#splash-quote') == 'our splash-quote' $ pytest -k test_homepage --browser chromium --browser firefox --verbose tests/test_browser.py::test_homepage[chromium] PASSED…

It would be neat if I could get coverage from these tests using python, but other than that I agree. Playwright is great.
Post reply on HN