Earlier quoted context omitted.
If it's a video site, I expect the video to play when I land, e.g. youtube. I'm initiating on purpose by browsing
The first thing I do when hitting a YouTube URL is stop the video. Then I'll either run youtube-dl on the URL, or just paste it straight into a proper video player (VLC).
It is not possible to detect and block Chrome headless
151–160 of 174 posts
Re: It is not possible to detect and block Chrome headless
#152Earlier quoted context omitted.
Poisoning the well is very effective. We employed it at a large ecommerce company that was getting hit by carders testing credit cards on low price point items(sub $5). We were playing cat and mouse with them for six months. Found certain attributes about the browser that the botnet was using and fed them randomized success/fail responses. After two weeks of feeding them bad data, they left and never came back. They…
The whole cat and mouse game thing... for some strange reason that sounds fun to me. Probably because I don't know the details and the workload involved in actually doing it (and it's not my money or inventory at stake). It seems like it would be exciting, in that somewhat naive juvenile-ish fantasy sort of way, to try and figure out how to mitigate the threat, implement it quickly, and deploy it to watch it play out…
Re: It is not possible to detect and block Chrome headless
#153Earlier quoted context omitted.
I dont stop crawlers, I only randomly feed damaged/wrong data to crawlers. I especially loving doing this for e-commerce sites. Now the table has turned. Try guess which fraction of your scrapped data were wrong.
My point is you’re not detecting the ones that you should be the most concerned with.
Re: It is not possible to detect and block Chrome headless
#154Earlier quoted context omitted.
I wonder how long it will be before someone comes up with the idea of using iPhone style facial recognition to tell whether a human is looking at the TV/Monitor or not.
Oh please no, exposing those sorts of APIs will quickly be utilized by ad-tech guys to make interstitial video ads that don't go away until you finish watching them.
Re: It is not possible to detect and block Chrome headless
#155Earlier quoted context omitted.
And I can attest that they often presume that I'm a robot. At this point it would be easier for me to write an alternative frontend to Google search (or just use duckduckgo), but it was be amusing to think that I might evade this by writing a script to simulate mouse movements to appear less robotic.
> And I can attest that they often presume that I'm a robot. In my experience this occurs when you are either doing this too much, or you are not accepting their cookies when logged in. (I don't recall the behavior when logged out.)
Actually, for whatever reason, this also often seems to cause reCapcha to essentially hellban me and just keep asking for me to solve an endless series of capchas. :/
Re: It is not possible to detect and block Chrome headless
#156Earlier quoted context omitted.
In my experience, the most effective counter measure to scraping is not to block, but rather to poison the well. When you detect a scraper - through what ever means - you don't block it, as that would tip it off that you are on to it. Instead you begin feeding plausible, but wrong data (like, add a random number to price). This will usually cause much more damage to the scraper than blocking would. Depending on your…
Poisoning the well is very effective. We employed it at a large ecommerce company that was getting hit by carders testing credit cards on low price point items(sub $5). We were playing cat and mouse with them for six months. Found certain attributes about the browser that the botnet was using and fed them randomized success/fail responses. After two weeks of feeding them bad data, they left and never came back. They…
Re: It is not possible to detect and block Chrome headless
#157Earlier quoted context omitted.
Poisoning the well is very effective. We employed it at a large ecommerce company that was getting hit by carders testing credit cards on low price point items(sub $5). We were playing cat and mouse with them for six months. Found certain attributes about the browser that the botnet was using and fed them randomized success/fail responses. After two weeks of feeding them bad data, they left and never came back. They…
The whole cat and mouse game thing... for some strange reason that sounds fun to me. Probably because I don't know the details and the workload involved in actually doing it (and it's not my money or inventory at stake). It seems like it would be exciting, in that somewhat naive juvenile-ish fantasy sort of way, to try and figure out how to mitigate the threat, implement it quickly, and deploy it to watch it play out…
Quite a while ago, I was involved in baiting an attacker in a somewhat different way, but with the same goal (destroying the value we were providing to them). After the attacker figured out what was going on, they issued a somewhat credible threat to damage the company's machines (they included some details demonstrating they had access to a couple internal machines at some point), attacked and DOSed them, and persistently tried spearphishing them for months afterwards.
I guess I'd just say, (a) doing this sort of thing responsibly sucks a lot of the fun out of it, and (b) don't underestimate the risks of things going pear-shaped. You could be buying yourself a lot of ongoing grief. Something as everyday as that spearphishing attack can be nerve wracking - even after making annoying the crap out of everyone by repeating how to be careful with email, there's no way to be sure it won't hit, and the next thing you know people are sitting around with upper management having conversations nobody wants to have about network segmentation and damage mitigation.
Re: It is not possible to detect and block Chrome headless
#158The EME DRM is part of the game for those who really want to block headless. It will arrive, sooner or later.
I always assumed DRM would eventually factor into this. I’ve only ever read about it in the context of media, but I’m assuming there’s ways to use it creatively for fingerprinting and blocking scraping as well. Do you have any links with insights into that?
Re: It is not possible to detect and block Chrome headless
#159Earlier quoted context omitted.
I think this is a good example of "poisoning the well" in practice, and I was in a similar position as you describe when I was working in incident response at a consumer bank a few years ago. That said, this is a very particular scenario, and I don't think you can generalize the effectiveness of the technique from this example. In situations where attackers are looking for boolean responses, i.e. to verify email addr…
Man. I really like that idea, but is giving a random "success" to a supposed credit card charge PCI compliant?
Most e-commerce sites will use a payment processor such as Braintree or Stripe so they don't even have to deal with PCI.
PCI applies to entities who store and process their own payments, e.g. Target, Amazon.
Re: It is not possible to detect and block Chrome headless
#160If you want to detect if a human is visiting your site, open an ad popup with a big close button directly over the content. A human being will always, 100% of the time, immediately close the popup. Automation won't care.
OK, but that is guaranteed to annoy users. Plus, I think you’re underestimating the intelligence of the people writing scrapers — obviously they’re going to visit the site manually and see what appears to be a fingerprinting measure. Then they’ll update the scraper to close that pop up. There are no effective solutions to this problem.