[deleted]
Someone is running mass vulnerability scans, spoofing AI bots like ClaudeBot
71–80 of 243 posts
Re: Someone is running mass vulnerability scans, spoofing AI bots like ClaudeBot
#72I recently blogged about some Cloudflare Workers I developed to combat this type of traffic: https://code.backwater.systems/blog/#2026-06-29T23:40:00.000...
Using a normal page per blog entry would go a long way to making your site more indexable, readable, shareable and seo-able. (Good article btw).
Re: Someone is running mass vulnerability scans, spoofing AI bots like ClaudeBot
#73Many of those user-agents listed are often faked. Look up which ASN owns their IP. If I block most VPS providers most of the faked bots vanish. There are still some running from residential and phones using hijacked code (readers that are not really just readers but really multipurpose proxies) . On that note, do not trust the linked source code but rather decompile the live code your phone is running and have AI ana…
Is there an easy way to block any requests originating from VPS etc instead of residential/commercial IP from legitimate users ? I know cloudflare does a few things but I really want to figure out a way to block any request say at nginx or caddy (reverse proxy) from reaching origin servers if they are not from an IP that is not a VPS etc.
There are several methods. [1] The most aggressive method-02 and method-03 on my document will block VPS and some data-centers but that also means it will block some legit users that are on a VPN. Most VPNs transit a data-center. If experimenting with these methods use a test server that you do not care about and set up a dummy site and ask people in your circle of friends to test it. I have to step away for a bit but if you have questions I will try to answer.
[1] - https://nochan.net/b/Internet-Crap/20260606-How-To-Block-Som...
Re: Someone is running mass vulnerability scans, spoofing AI bots like ClaudeBot
#74Earlier quoted context omitted.
You are the 0.001%
Much more than 0.001% of people care about their privacy or (the larger portion) do not have unfiltered access to the internet.
I second this. When I have tested blocking VPS/data-centers to my silly blog there were about a dozen people on HN [1] that could not view my site out of the roughly ~17,000 (not counting bots) that could. It's not a big number but those are real people and they count.
I am going to move full blocking to a test node that people can play with but I have to finish working with Claude to revise someones repo is is no longer maintained because one does not simply put an anonymous chan board on the great wide open internets without some critical thinking.
Re: Someone is running mass vulnerability scans, spoofing AI bots like ClaudeBot
#75Why would you voluntarily pretend to be a AI bot, when those have already a much higher chance of being blocked? Seems holly unproductive. Best hypothesis I can come up with is to somehow make the AI companies look bad, but they seem to be doing an excellent job at that themselves already by scraping everyone hundreds of times per hour over and over.
Because businesses dont want them blocked, that would be a very stupid thing for most of them to do given its becoming a vital traffic source now that people are using chatbots instead of google.
Re: Someone is running mass vulnerability scans, spoofing AI bots like ClaudeBot
#76Earlier quoted context omitted.
Is there an easy way to block any requests originating from VPS etc instead of residential/commercial IP from legitimate users ? I know cloudflare does a few things but I really want to figure out a way to block any request say at nginx or caddy (reverse proxy) from reaching origin servers if they are not from an IP that is not a VPS etc.
Yes but it’s not cheap. Maxmind and ipinfo etc sell a tier that tells you this information, then you can 403 based on it. But the price is nuts like $40,000 a year.
Open source works just fine for this - take a look at geoacumen-country
Re: Someone is running mass vulnerability scans, spoofing AI bots like ClaudeBot
#77Just in case any of the authors read HN, I'm getting a pretty crazy rendering bug on this page, where a bunch of the contents are redrawing up and down by a few pixels. It seemed to go away with resizing the width a few times, but I didn't look into it too hard. My page width was probably small on first draw. Incredibly distracting though and hard to read with the text moving. Using latest chrome, and it occurred on more than one page refresh. I didn't dig in beyond that though.
Re: Someone is running mass vulnerability scans, spoofing AI bots like ClaudeBot
#78Just in case any of the authors read HN, I'm getting a pretty crazy rendering bug on this page, where a bunch of the contents are redrawing up and down by a few pixels. It seemed to go away with resizing the width a few times, but I didn't look into it too hard. My page width was probably small on first draw. Incredibly distracting though and hard to read with the text moving. Using latest chrome, and it occurred on…
I'm seeing the same thing on Firefox on Linux. It almost looks like the page scroll is jiggling up and down a tiny amount constantly when it's supposed to be stationary.
Re: Someone is running mass vulnerability scans, spoofing AI bots like ClaudeBot
#79Earlier quoted context omitted.
I work for IPinfo. We offer IPinfo Lite for free. With a little bit of time in identifying the ASNs, you can implement a decent way to block a good number of bots fairly easily using the free data alone.
Good to know, thank you. Would you do this by fully blocking particular ASNs? Or something more granular?
You can block entire ASNs. If you are frustrated with bots, blocking Tencent's entire IP address space would have very few downsides.
If you have fail2ban or NGINX logs, you can use our CLI to summarize those IPs and identify the ASNs you want to block. But before you block entire ASNs, make sure they are not classified as "ISP" type. For that, visit our website's ASN page first.
I have quite a few community posts around this approach. https://community.ipinfo.io/
If you have raw logs, you can send them to me as well, and I can review them and provide some guidance.
Re: Someone is running mass vulnerability scans, spoofing AI bots like ClaudeBot
#80Earlier quoted context omitted.
Is there an easy way to block any requests originating from VPS etc instead of residential/commercial IP from legitimate users ? I know cloudflare does a few things but I really want to figure out a way to block any request say at nginx or caddy (reverse proxy) from reaching origin servers if they are not from an IP that is not a VPS etc.
There are several methods. [1] The most aggressive method-02 and method-03 on my document will block VPS and some data-centers but that also means it will block some legit users that are on a VPN. Most VPNs transit a data-center. If experimenting with these methods use a test server that you do not care about and set up a dummy site and ask people in your circle of friends to test it. I have to step away for a bit bu…
Your site does not currently seem to be reachable / responsive when I try to reach it from a US Comcast IPv4 address - you are not advertising IPv6.
Edit / Update: It was Apple's Private browsing mode that causes it not to respond. I can now see it when this is disabled.