71–80 of 158 posts
I expect >99% of my web traffic is bots or agents and I was actually considering removing the page counter as it is pretty meaningless and makes my site look far busier than it is. I am reluctant to however do anything about it just in case it accidentally stops a genuine human reading it or downloading my books.
Nice, I will have to take a look at some point.
Earlier quoted context omitted.
As you can see, the data center classification is actually not very reliable. Actual bad actors rent residential-classified connections. Good bots don't.
I can spot them but the feds are working on taking down the biggest residential proxy malware nodes. It's a game of whack-a-mole but I will let them play that game. There are some easy ways to spot some of them but I will save that for another article.
I say this as someone who runs several publicly-facing websites, and also scrapes a few others for use in my own tools: why do people care so much about bots? WordPress with a cache can handle on the order of 1k requests per second on the cheapest VPS money can buy. An actual proper static site would likely 10x that. Are you using lambdas or something like that to serve a blog? Is it basically just OCD? Vulnerability…
For me it's just a fun exercise to see what generalizations I can make about bots that botters can not really avoid or code around such as javascript . Yeah this blog is all precompressed static content in a ram disk. I'm sure it could handle a few hundred thousand requests per second. This is to show methods people could use on their forums, chan sites, chat servers, etc... Every option is optional and tunable. Any…
I like the idea of adding a fake cpanel subdomain for 169.254.169.254 so that script kiddies will start port-scanning their own hosting provider, which will likely get them flagged/banned.
The guy is certainly kind of visibly ravaged by the more shady denizens of the global internet. But there are some fun things to read there in any case.
That is both funny, sad and oddly appropriate. Indeed, decades of running forums, chan and chat servers. I came here to take a break from all of that. It's been a nice vacation so to speak, letting someone else do all the management of a user-content driven site. Now it's all just tinkering and experimenting for me.
Earlier quoted context omitted.
> Obvious proxy is obvious. > if ($http_x_forwarded_for) {.... This may block schools and libraries that use content blockers. Often the internal client is left to make abuse tracking easier (or because the overworked admin didn't know they could turn it off).
This may block schools and libraries Oh, well that is ok for me I suppose. I add RTA/adult headers that hopefully they also look for and block using parental controls as adult content should not be viewed in a school or library. I could add a note suggesting to skip that step if one wishes schools and libraries that may be using a proxy to view.
I say this as someone who runs several publicly-facing websites, and also scrapes a few others for use in my own tools: why do people care so much about bots? WordPress with a cache can handle on the order of 1k requests per second on the cheapest VPS money can buy. An actual proper static site would likely 10x that. Are you using lambdas or something like that to serve a blog? Is it basically just OCD? Vulnerability…
Yes we know - I still had my site knocked out a couple of times by overeager bots. They can be really abusive.
Earlier quoted context omitted.
I can spot them but the feds are working on taking down the biggest residential proxy malware nodes. It's a game of whack-a-mole but I will let them play that game. There are some easy ways to spot some of them but I will save that for another article.
Not all residential proxies are malware. You can get paid crypto to offer up your connection. Prices will go up because of the malware crackdowns but the industry will still exist.
Voluntary malware? Funded malware facilitation?
I'm not against crypto, but it would sound more legit if you removed the word crypto from your sentence. If it's exclusively the ability to be paid in crypto, as opposed to local currency or USD, then I'd guess it leans pretty heavily to shady-ness.
I pretty much think all residential proxy traffic is undesirable - otherwise it wouldn't need to use a residential proxy (they're used to get around erstwhile blocking, which someone has actively chosen to put in place - rightly or wrongly).