231–232 of 232 posts
Earlier quoted context omitted.
This is a very obvious thing to say. Perhaps it's needed to be said, I don't know -- It's just a very obvious counter.
Yeah, it seemed obvious... but judging by all of the comments here on how to "bypass" 403s, it actually wasn't obvious at all.
Earlier quoted context omitted.
Virtually everything can be easily defeated. The only outfit I've consistently seen put up a good fight is Distil. They do it by acting a little like Cloudflare. They put their servers in front of your www facing endpoints and use ML to mine their global client traffic to identify bot signals (aided by some aggressive in-browser javascript fingerprinting).
Yeah, Distil is the first outfit I've encountered where they've got the model to make it really hard to reliably bypass. It comes down to "I can spend a significant amount of time trying to bypass this, and I would, but they would likely identify and block me again within a few weeks at most.", and it's not worth it when it's only part of what I need to do to scrap some data, and it's their entire job, and they can a…