21–28 of 28 posts
This was after web had to geo block China & Singapore some weeks earlier.
These AI scraping guys are destroying the web for normal folks in these countries where they run data scrapers.
> hundreds of requests per day Does this matter? I can handle hundreds of requests per day with no issue on a home cable modem connection and my desktop pc running nginx. In fact I do and have since the 56k days. With an actual server or VPS with a big pipe in a datacenter this should literally be below noticing in terms of cost. I would characterize this response to normal public website traffic as more harmful than…
Earlier quoted context omitted.
It’s possible, but I think it’s typically used for ingress (ie same IP, but multiple destinations, follow BGP to closest one). I don’t think I’ve seen a similar case for anycast egress. Naively, doesn’t seem like it would work well because a lot of the internet (eg non-anycast geographic load balancing) relies on unique sources, and Cloudflare definitely break out their other anycast addresses (eg they don’t send out…
Cloudflare actually does anycast for egress too, if that is what you meant: https://blog.cloudflare.com/cloudflare-servers-dont-own-ips-...
Tunneling shenanigans are fine for ACKs, but it's inefficient and therefore pretty unlikely that they are doing this for ingress object traffic.
What you're seeing is normal bot behaviour, they constantly scan every website for different purposes. 100–500 requests per IP is nothing you should worry about or take any action against.
tirreno works on the backend, so sometimes we use it to analyze bot behaviour when they start doing something really suspicious, like massive requests (hundreds of thousands a day) or scanning all possible files/folder structures, which could easily result in half a million requests in short period of time.
We’ve been experiencing the same thing. On further inspection, we discovered that the owner of the data centers was Tencent. So we blocked them at the ASN level across countries. This was after web had to geo block China & Singapore some weeks earlier. These AI scraping guys are destroying the web for normal folks in these countries where they run data scrapers.
It's like if you had incidents with a few violent drunk Brovanians in your town, then saying it's those few peoples fault that Brovanians are now being discriminated against and are being banned from entering shops just because they come from the same place as the vandals.
Site operators arbitrarily blocking entire countries due to a few botters (albeit with a lot of bots) causing issues aren't without responsibility in the loss of an open web.
You have a choice in how to respond and where to draw lines. We can't just throw up our hands and blame the botters.
block from any to 173.245.58.0/24
# US https://www.abuseipdb.com/check/173.245.58.143
block from any to 173.245.58.140
# US https://www.abuseipdb.com/check/173.245.58.143
block from any to 173.245.58.143
# US https://www.abuseipdb.com/check/173.245.58.151
block from any to 173.245.58.151
# US https://www.abuseipdb.com/check/173.245.58.165
block from any to 173.245.58.165
> hundreds of requests per day Does this matter? I can handle hundreds of requests per day with no issue on a home cable modem connection and my desktop pc running nginx. In fact I do and have since the 56k days. With an actual server or VPS with a big pipe in a datacenter this should literally be below noticing in terms of cost. I would characterize this response to normal public website traffic as more harmful than…
> I can handle hundreds of requests per day with no issue on a home cable modem connection and my desktop pc running nginx. And what kind of ecommerce site are you running on that nginx? First thing that get overwhelmed by bot traffic is DB. With a tiny one, with low total connection limit and bots hitting less common path like browsing 20th page of product search results, it is really easy to get DoS. I remember hav…
The multiple locations are just showing which Cloudflare POP handled each request (ORD, SJC, LAX = their data centers). That's expected behavior when you're proxied through CF.
Check the CF-Connecting-IP header to get the real visitor IP. What you're logging right now is basically "which Cloudflare server talked to your origin," not "where the bot actually is."