Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Playwright: Automate Chromium, WebKit and Firefox

github.com

Post reply on HN
Print view View original

31–40 of 149 posts

Re: Playwright: Automate Chromium, WebKit and Firefox

#31

Post by naasking »

tmcneal wrote: post #19
naasking wrote: post #3

Anyone have experience with Playwright compared to Selenium? I have a fairly large test suite and Selenium produces constant false positive errors, typically due to various timeouts that seem fundamentally unsolvable when running it from .NET. It's just very finicky. I don't know if it's Selenium specifically or some problem with the .NET binding, but I figure Microsoft must have better .NET integration so it will at…

I'm not sure if any of these are pertinent to your tests, but these are the issues I see most often that cause flaky tests: - Hard-coded waits in your code, like "Thread.sleep(1000)". A better alternative is to replace hard-coded waits with something that waits for an element or value to appear on the page. i.e. click on a button and wait for a 'Success' message to appear. Puppeteer and Playwright both have good cons…

> - Hard-coded waits in your code, like "Thread.sleep(1000)". A better alternative is to replace hard-coded waits with something that waits for an element or value to appear on the page.

We don't do any timed waits, all of our waits are for an element or value to appear, but these waits never complete sometimes, non-deterministically. We then added a long 5 min timeout on these waits because we know the test will never complete at that point. It's always fine in manual testing though, and if we don't run the browser in headless mode and watch it work. Very frustrating.

Sometimes the HTTP requests themselves timeout after a few minutes, but this never happens in manual testing either. That's actually the most common issue these days, and this happens non-deterministically too. This is what I meant by "flaky".

Re: Playwright: Automate Chromium, WebKit and Firefox

#32

Post by tempest_ »

headlessvictim2 wrote: post #27
cmeacham98 wrote: post #22

Earlier quoted context omitted.

100 requests/second isn't that much, especially if you're fronting your website with Cloudflare. Do you have some unauthenticated endpoint(s) that eat up a ton of server CPU?

Thanks for the reply! The freemium service provides access to machine learning models on GPU instances, served with FastAPI. Each request invokes a compute-intensive ML model, but perhaps there is something wrong with the FastAPI configuration as well?

It could be.

I watch the FastAPI repos a lot and tones of people do not understand how async python works and put their models with sync code in an async context.

Re: Playwright: Automate Chromium, WebKit and Firefox

#33

Post by twohaibei »

johnnypangs wrote: post #10

What do people thing of playwright vs cypress? I've been considering using playwright instead as it supports more browsers and I feel like it's easier to do production monitoring (by putting it in a aws lambda or using checkly) - Cypress: https://www.cypress.io - Playwright aws lambda: https://github.com/PauloGoncalvesBH/running-playwright-on-aw... - Checkly: https://www.checklyhq.com

I highly recommend codeceptjs. After 4 years of using test cafe for e2e testing, codecept has proven to be much more pleasant to use.

Re: Playwright: Automate Chromium, WebKit and Firefox

#34

Post by darkstar999 »

austincheney wrote: post #23
headlessvictim2 wrote: post #20

Off-topic, but our freemium website is under attack by headless browsers. The freemium service provides access to compute-heavy machine learning models running on GPUs. Hackers blast 50-100 requests in the same second, which clog the servers and block legitimate users. We reported IPs to AWS and use Cloudflare "Super Bot Fight Mode" to thwart attacks, but the hackers still break through. We don't require accounts, bu…

Browser automation will occur by executing events in the DOM or by calling properties of the page/window. It’s all JavaScript designed for user interaction executed by a bot. The one event that cannot be automated is cursor movement/position. Put a check into your event handlers that check that the cursor is actually over the event target.

That sounds like an accessibility problem.

Re: Playwright: Automate Chromium, WebKit and Firefox

#35

Post by vthommeret »

Reposting my previous notes on Playwright (https://news.ycombinator.com/item?id=30060135):

I just want to plug Playwright by Microsoft as I've been using it over the past month and have had a really great experience with it: https://playwright.dev It's built by the founders of Puppeteer which came out of the Chrome team. Some things I like about it:

1. It's reliable and implements auto-waiting as described in the article. You can use modern async/await syntax and it ensures elements are a) attached to the DOM, visible, stable (not animating), can receive events, and are enabled: https://playwright.dev/docs/actionability

2. It's fast — It creates multiple processes and runs tests in parallel, unlike e.g. Cypress.

3. It's cross-browser — supports Chrome, Safari, and Firefox out-of-the-box. 4. The tracing tools are incredible, you can step through the entire test execution and get a live DOM that you can inspect with your browser's existing developer tools, see all console.logs, etc...

5. The developers and community are incredibly responsive. This is one of the biggest ones — issues are quickly responded to and addressed often by the founders, pull requests are welcomed and Slack is highly active and respectful.

My prior experience with end-to-end tests was that they were highly buggy and unreliable and so Playwright was a welcome surprise and inspired me to fully test all the variations of our checkout flow.

Re: Playwright: Automate Chromium, WebKit and Firefox

#36

Post by headlessvictim2 »

tempest_ wrote: post #32
headlessvictim2 wrote: post #27

Earlier quoted context omitted.

Thanks for the reply! The freemium service provides access to machine learning models on GPU instances, served with FastAPI. Each request invokes a compute-intensive ML model, but perhaps there is something wrong with the FastAPI configuration as well?

It could be. I watch the FastAPI repos a lot and tones of people do not understand how async python works and put their models with sync code in an async context.

Consider us one. :)

We tried removing "async" -- thinking it would force sequential processing -- but it unexpectedly seemed to cause parallel processing of requests, which caused CUDA memory errors.

Before removing "async", this is the weird behavior we observed:

* Hacker blasts 50-100 requests.

* Our ML model processes each request in normal time and sequentially.

* But instead of returning individual responses immediately, the server holds onto all responses -- sending responses only when the last request finishes (or a bunch of requests finish).

* Normally, request 1 should return in N seconds, request 2 in 2N seconds, but with this, all requests returned in about N50 seconds (assuming batch size of 50).

1. Any suggestions on this?

2. Mind clarifying how sync vs aync works? The FastAPI docs are unclear.

Any help would be much appreciated.

This has been extremely frustrating.

Re: Playwright: Automate Chromium, WebKit and Firefox

#37

Post by austincheney »

headlessvictim2 wrote: post #30
austincheney wrote: post #23

Earlier quoted context omitted.

Browser automation will occur by executing events in the DOM or by calling properties of the page/window. It’s all JavaScript designed for user interaction executed by a bot. The one event that cannot be automated is cursor movement/position. Put a check into your event handlers that check that the cursor is actually over the event target.

This is interesting. Thanks for sharing. Are you saying block form submission unless the cursor is over the event target? If so: * How to handle legitimate requests from mobile users? * How to handle form submissions with the "return" key?

Mobile users will use touch events instead of click events and likely your interface will be different and the screen width will be different. Check for these things along with keywords from the user agent string to determine mobile users from other users.

Return key on a control in a form will fire a submit event. Check for cursor position in your submit handler.

Re: Playwright: Automate Chromium, WebKit and Firefox

#38

Post by austincheney »

darkstar999 wrote: post #34
austincheney wrote: post #23

Earlier quoted context omitted.

Browser automation will occur by executing events in the DOM or by calling properties of the page/window. It’s all JavaScript designed for user interaction executed by a bot. The one event that cannot be automated is cursor movement/position. Put a check into your event handlers that check that the cursor is actually over the event target.

That sounds like an accessibility problem.

Use an alternate control for keyboard navigation that is visually hidden and is accessed only by tab focus.

Re: Playwright: Automate Chromium, WebKit and Firefox

#39

Post by syspec »

Interesting tidbit:

One of the main contributors of this project[0], was the core contributor (creator?) of Puppeteer[1], but then I guess left Google to join Microsoft and work on this[2][3].

[0] - https://github.com/aslushnikov

[1] - https://github.com/puppeteer/puppeteer/

[2] - https://github.com/microsoft/playwright/graphs/contributors

[3] - https://github.com/microsoft/playwright/graphs/contributors

Re: Playwright: Automate Chromium, WebKit and Firefox

#40

Post by rabi_molar »

headlessvictim2 wrote: post #20

Off-topic, but our freemium website is under attack by headless browsers. The freemium service provides access to compute-heavy machine learning models running on GPUs. Hackers blast 50-100 requests in the same second, which clog the servers and block legitimate users. We reported IPs to AWS and use Cloudflare "Super Bot Fight Mode" to thwart attacks, but the hackers still break through. We don't require accounts, bu…

Perhaps Captcha?
Post reply on HN