21–30 of 32 posts
I've used Puppeteer extensively and its the most unstable tool that I've come across in recent years. Its full of race conditions and the waitForSomething functions will sometimes work, sometimes not. It's usable, but it's very aggravating and uncomfortable to use.
Have you tried Playwright? It is a fork of Puppeteer that adds auto-waiting
I've used Puppeteer extensively and its the most unstable tool that I've come across in recent years. Its full of race conditions and the waitForSomething functions will sometimes work, sometimes not. It's usable, but it's very aggravating and uncomfortable to use.
Have you tried Playwright? It is a fork of Puppeteer that adds auto-waiting
That's the best one. Of all the browser automators, playwright is the best. Never had a wait fail or anything like that. maybe I just got lucky, but if you're looking to do something with browser automation, try Playwright first, then look elsewhere
I've used Puppeteer extensively and its the most unstable tool that I've come across in recent years. Its full of race conditions and the waitForSomething functions will sometimes work, sometimes not. It's usable, but it's very aggravating and uncomfortable to use.
Surprising to me -- I used puppeteer last year for a couple of projects and found it to be extremely pleasant to use -- almost fun even! You can just pick it up and most of the api seemed relatively straightforward. I found it a little annoying to get things like reliable window placement (yes I do want the chrome incognito to be in this exact place and size everytime). But it seemed very solid overall and especially…
I've never used Puppeteer, but having used a lot of US-hosted web services from Asia, I've seen plenty of latency-sensitive bugs (or at least annoyances not present for NYC-based users).
Earlier quoted context omitted.
Surprising to me -- I used puppeteer last year for a couple of projects and found it to be extremely pleasant to use -- almost fun even! You can just pick it up and most of the api seemed relatively straightforward. I found it a little annoying to get things like reliable window placement (yes I do want the chrome incognito to be in this exact place and size everytime). But it seemed very solid overall and especially…
The GP is describing lots of timing-related issues. It's possible you experience much lower latency than the GP. I've never used Puppeteer, but having used a lot of US-hosted web services from Asia, I've seen plenty of latency-sensitive bugs (or at least annoyances not present for NYC-based users).
Different cdn behaviors and differences in the way bot detection logic gets triggered due to geoip ip categorization lists (which definitely impact a lot of sites, often in subtle ways when - say - only a subset of the javascript assets on a page fail to load).
In my experience though, these observations are just as likely to be programmer error as they are to be legitimate claims that 'my code doesn't work because of timing issues in the puppeteer implementation.' The programming model that puppeteer exposes tends toward the rational in my opinion.
Browser automation often involves asynchronously instructing the page to simulate a click on something in order to induce some javascript in a page to run some effect, in order to repaint the dom -- and this kind of logic chain will _always_ demonstrate timing variation. But the puppeteer api's make it pretty straightforward to sequence the events that are simulated so that you can ensure you issue a click on something that exists or a waitForSomethingToExist polling based detector that will trigger reliably when a given dom node appears ... Depending on the specific page's implementation, sometimes this kind of logic can be harder to write for one page than another -- but my first guess for the reason why a given page might make this kind of logic flakey or hard to implement is more likely to be an obscure implementation detail of the page than a puppeteer bug (as a first approximation).
Could you elaborate a little on ‘puppeteer in the browser’? Is puppeteer running on a webserver then the repl connecting to it? Or is puppeteer completely contained within each users browser?
Sure! Puppeteer is a node-based library, and pretty much all the web-apps out there that let you run puppeteer code do it in an elaborate node sandbox. This tool gets around that by running puppeteer in your own browser, making it a lot faster and more secure
I’m curious if it’s possible to proxy the network requests so for example it would use the browsers IP address instead of the server?
I've used Puppeteer extensively and its the most unstable tool that I've come across in recent years. Its full of race conditions and the waitForSomething functions will sometimes work, sometimes not. It's usable, but it's very aggravating and uncomfortable to use.
Surprising to me -- I used puppeteer last year for a couple of projects and found it to be extremely pleasant to use -- almost fun even! You can just pick it up and most of the api seemed relatively straightforward. I found it a little annoying to get things like reliable window placement (yes I do want the chrome incognito to be in this exact place and size everytime). But it seemed very solid overall and especially…
Earlier quoted context omitted.
The GP is describing lots of timing-related issues. It's possible you experience much lower latency than the GP. I've never used Puppeteer, but having used a lot of US-hosted web services from Asia, I've seen plenty of latency-sensitive bugs (or at least annoyances not present for NYC-based users).
There are all sorts of annoyances that can manifest differently in different parts of the world when doing various forms of browser automation from different parts of the world. Different cdn behaviors and differences in the way bot detection logic gets triggered due to geoip ip categorization lists (which definitely impact a lot of sites, often in subtle ways when - say - only a subset of the javascript assets on a…
Earlier quoted context omitted.
can you explain how this is architected? you are running a browser on a server and its streaming the video to the canvas? where does the cloudflare bit come in? very interesting project!
You’ve got it: the core service listens for inbound Upgrade http requests, starts a browser, then funnels the connection into the browser. This debugger simply sits on top of all that, and puts the code/execution context in the browser versus the server. Cloud flare is simply detecting that our server IP is a known headless chrome instance, and is serving their bot detection check