if you want to block scrapers, just add rate limiting...
I'll eat you through a proxy network then, unless you want to slow down your legitimate users too.
It is not possible to detect and block Chrome headless
11–20 of 174 posts
Re: It is not possible to detect and block Chrome headless
#12if you want to block scrapers, just add rate limiting...
while ScraperBlocked:
scraper.sleep(5)Re: It is not possible to detect and block Chrome headless
#13"That’s when it becomes impossible. You can come up with whatever tests you want, but any dedicated web scraper can easily get around them." As long as the logic is hidden from the scrapers, i.e. not running in a web browser, scrapers are at a disadvantage. They don't have the data about the users that websites have. And even something as simple as Accept-Language header associated with an IP subnet is a data point t…
The counterpoint (and what this article mentioned in several places) is that the "more aggressive fingerprinting" techniques can have high false positive rates, and then the legitimate users of your site are going to end up thinking it's broken or the data never loads because they're inadvertently triggering your fingerprinting.
It feels pretty ridiculous to tell a user "you can't use our site because your system language settings don't match our IP list for that area."
Sidestepping the fact that assuming you can tell someone's language from the geo mapping of their IP address is already pretty problematic.[0]
[0]: https://medium.com/@kristopolous/stop-guessing-languages-bas...
Re: It is not possible to detect and block Chrome headless
#14if you want to block scrapers, just add rate limiting...
There is a multi-million dollar industry around blocking scrapers, it's not "just add rate limiting".
Re: It is not possible to detect and block Chrome headless
#15if you want to block scrapers, just add rate limiting...
That's old hat and ineffective. Scrapers usually proxy through large lists of rotating ip addresses. There are lots of services for it.
Re: It is not possible to detect and block Chrome headless
#16The problem is that you can easily detect that some properties have been overloaded. For example, you can execute Object.getOwnPropertyDescriptor(navigator, "languages") to detect if navigator.languages is a native property or not.
Re: It is not possible to detect and block Chrome headless
#17The EME DRM is part of the game for those who really want to block headless. It will arrive, sooner or later.
Re: It is not possible to detect and block Chrome headless
#18Good. It shouldn't.
The web should be open the fact that people are still trying to stop this is a joke.
Re: It is not possible to detect and block Chrome headless
#19Good, the less effective various spying techniques are, and the easier they are to throw off, the better the internet is for its users. I don't want any website owners to know what device, browser, or other program, I use to access their site, and they have no business knowing that. I like it being a piece of information I can supply voluntarily for my own purposes, and I get the heebie jeebies every time I read abou…
This incentivizes more aggressive fingerprinting, not the other way around. Too bad people don't realize it.
Browser fingerprinting, I almost forgot. Non aggressive and impossible to stop.
Re: It is not possible to detect and block Chrome headless
#20The problem is that you can easily detect that some properties have been overloaded. For example, you can execute Object.getOwnPropertyDescriptor(navigator, "languages") to detect if navigator.languages is a native property or not.
Can’t I accomplish the same thing by compiling a modified version of headless chrome?