Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Playwright: Automate Chromium, WebKit and Firefox

github.com

Post reply on HN
Print view View original

1–10 of 149 posts

Re: Playwright: Automate Chromium, WebKit and Firefox

#2

Post by simonw »

Electron appear to have dropped support for their previous automated testing framework Spectron - https://github.com/electron-userland/spectron/blob/master/RE... - and now suggest Playwright as an alternative: https://www.electronjs.org/docs/latest/tutorial/automated-te... and https://playwright.dev/docs/api/class-electronapplication/

Re: Playwright: Automate Chromium, WebKit and Firefox

#3

Post by naasking »

Anyone have experience with Playwright compared to Selenium? I have a fairly large test suite and Selenium produces constant false positive errors, typically due to various timeouts that seem fundamentally unsolvable when running it from .NET. It's just very finicky.

I don't know if it's Selenium specifically or some problem with the .NET binding, but I figure Microsoft must have better .NET integration so it will at least eliminate that possible source of problems.

Re: Playwright: Automate Chromium, WebKit and Firefox

#4

Post by mattlondon »

naasking wrote: post #3

Anyone have experience with Playwright compared to Selenium? I have a fairly large test suite and Selenium produces constant false positive errors, typically due to various timeouts that seem fundamentally unsolvable when running it from .NET. It's just very finicky. I don't know if it's Selenium specifically or some problem with the .NET binding, but I figure Microsoft must have better .NET integration so it will at…

I have had similar issues with selenium via other languages too - it is generally pretty flaky. E.g. saying a button or some other element doesn't exist when it clearly does.

With great care and effort you can make your tests reliable (especially if you are happy to allow a "best of 3" type test strategy to allow for 1 flake and 2 passes) though. Prodigious use of the wait (i.e. stdlib polling) primitives seems to give you the most bang for your buck.

I am note sure if this is just the nature of web automation, or if selenium is just crap? My gut is to say it is selenium's fault since we never get the same issues when using javascript in the DOM or in an extension)...maybe it is the browser APIs I guess? U have no idea but if this playwright is any better than that would be superb.

Re: Playwright: Automate Chromium, WebKit and Firefox

#5

Post by londons_explore »

naasking wrote: post #3

Anyone have experience with Playwright compared to Selenium? I have a fairly large test suite and Selenium produces constant false positive errors, typically due to various timeouts that seem fundamentally unsolvable when running it from .NET. It's just very finicky. I don't know if it's Selenium specifically or some problem with the .NET binding, but I figure Microsoft must have better .NET integration so it will at…

I think it's possible to write tests in selinium which are time-independant... Eg. "Wait for element #foo to exist".

You can also give the browser a virtual clock so that you can use time based timeouts and give every test a timeout of 3 hours, but those 3 hours only take milliseconds in real time. That approach gets CPU expensive if your site has any background polling scripts or animation, because obviously the animation will end up animating a lot during the test!

Re: Playwright: Automate Chromium, WebKit and Firefox

#6

Post by simonw »

naasking wrote: post #3

Anyone have experience with Playwright compared to Selenium? I have a fairly large test suite and Selenium produces constant false positive errors, typically due to various timeouts that seem fundamentally unsolvable when running it from .NET. It's just very finicky. I don't know if it's Selenium specifically or some problem with the .NET binding, but I figure Microsoft must have better .NET integration so it will at…

On paper Playwright should be a LOT better - it's taken a similar approach to Cypress, where everything is designed around the need to reduce flaky tests.

In Playwright that manifests itself as the "auto-wait" feature: https://playwright.dev/docs/actionability

You can do this kind of thing with Selenium too but it wasn't designed in from the very start of that project.

Re: Playwright: Automate Chromium, WebKit and Firefox

#7

Post by defied »

I’m working on a project that provides remote browsers, running on VMs/containers, capable of running Playwright tests (and Puppeteer scripts): https://headlesstesting.com/

We’ve seen a consistent growth of interest in people wanting to use Playwright for browser automation (and testing).

Re: Playwright: Automate Chromium, WebKit and Firefox

#8

Post by naasking »

londons_explore wrote: post #5
naasking wrote: post #3

Anyone have experience with Playwright compared to Selenium? I have a fairly large test suite and Selenium produces constant false positive errors, typically due to various timeouts that seem fundamentally unsolvable when running it from .NET. It's just very finicky. I don't know if it's Selenium specifically or some problem with the .NET binding, but I figure Microsoft must have better .NET integration so it will at…

I think it's possible to write tests in selinium which are time-independant... Eg. "Wait for element #foo to exist". You can also give the browser a virtual clock so that you can use time based timeouts and give every test a timeout of 3 hours, but those 3 hours only take milliseconds in real time. That approach gets CPU expensive if your site has any background polling scripts or animation, because obviously the ani…

> I think it's possible to write tests in selinium which are time-independant... Eg. "Wait for element #foo to exist".

Yes, this is what I've done but the elements non-deterministically do or do not appear according to selenium, and then the wait times out. This happens for dozens of tests across dozens of pages with no issue with manual use, so something fishy is going on.

Re: Playwright: Automate Chromium, WebKit and Firefox

#9

Post by i_like_apis »

naasking wrote: post #3

Anyone have experience with Playwright compared to Selenium? I have a fairly large test suite and Selenium produces constant false positive errors, typically due to various timeouts that seem fundamentally unsolvable when running it from .NET. It's just very finicky. I don't know if it's Selenium specifically or some problem with the .NET binding, but I figure Microsoft must have better .NET integration so it will at…

While I have not used Playwright (but have a lot of experience with Se), I would say the code style is refreshing:

    // Expect an element "to be visible".
    await expect(page.locator('text=Learn more').first()).toBeVisible();
Writing await for every action makes the timeout of the action seem more explicitly declared. There seems to be more granular control of timeouts as well https://playwright.dev/docs/test-timeouts

> I don't know if it's Selenium specifically or some problem with the .NET binding

If the execution in .NET is slow then I suppose it could be .NET. But it could be (and often is) the suite design. You must wait for /everything/ before interacting with it because the code execution is quicker than the page.

Large Se/Webdriver suites are often a PIA. I find it's nice to write them with Python or Ruby so they can be debugged interactively with the an interactive shell.

Re: Playwright: Automate Chromium, WebKit and Firefox

#10

Post by johnnypangs »

What do people thing of playwright vs cypress? I've been considering using playwright instead as it supports more browsers and I feel like it's easier to do production monitoring (by putting it in a aws lambda or using checkly)

- Cypress: https://www.cypress.io

- Playwright aws lambda: https://github.com/PauloGoncalvesBH/running-playwright-on-aw...

- Checkly: https://www.checklyhq.com

Post reply on HN