Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Someone is running mass vulnerability scans, spoofing AI bots like ClaudeBot

knownagents.com

Post reply on HN
Print view View original

171–180 of 214 posts

Re: Someone is running mass vulnerability scans, spoofing AI bots like ClaudeBot

#171

Post by inigyou »

Bender wrote: post #74
basilikum wrote: post #18

Earlier quoted context omitted.

Much more than 0.001% of people care about their privacy or (the larger portion) do not have unfiltered access to the internet.

I second this. When I have tested blocking VPS/data-centers to my silly blog there were about a dozen people on HN [1] that could not view my site out of the roughly ~17,000 (not counting bots) that could. It's not a big number but those are real people and they count. I am going to move full blocking to a test node that people can play with but I have to finish working with Claude to revise someones repo is is no lo…

There were a dozen people who reported not being able to access your site.

The complete hysteria people go to over the near-non-issue of bots is, well, completely hysterical. Don't be that guy.

Re: Someone is running mass vulnerability scans, spoofing AI bots like ClaudeBot

#172

Post by inigyou »

bflesch wrote: post #39
pixl97 wrote: post #26

Earlier quoted context omitted.

Problem here is there are not single fibers attaching (most) countries, but a bunch of them. If you control both the ingress and egress for some particular users it's possible, but if you don't then your probing packing may end up back in China with a lot of evidence of backscatter.

I'd be surprised if there is a single route from EU to non-EU countries which does not pass through British control.

Does Britain own all fiber links between Switzerland and France?

Re: Someone is running mass vulnerability scans, spoofing AI bots like ClaudeBot

#173

Post by inigyou »

Gigachad wrote: post #134
esskay wrote: post #75

Earlier quoted context omitted.

Because businesses dont want them blocked, that would be a very stupid thing for most of them to do given its becoming a vital traffic source now that people are using chatbots instead of google.

From what I have seen at work, everyone is using chat bots but no one is visiting websites through them. We still get almost all traffic through social media and google search.

Yeah. Do you want the bot to buy the product from your website or your competitors?

Re: Someone is running mass vulnerability scans, spoofing AI bots like ClaudeBot

#174

Post by inigyou »

dewey wrote: post #97
ChillyCapy wrote: post #87

Fake Googlebot visits are #1 in website logs I've been working on. At the beginning I was fighting with them using Cloudflare ASN block rules or their managed Bot Fight mode but it appeared to be not only pointless, but also harmful for my websites. Bot Fight mode randomly started blocking real Bing / Google / OpenAI crawlers what wasted crawling budget and discouraged crawlers to revisit updated pages. Sometimes it'…

For Google it's pretty straight forward to throw away fake crawlers by just only allowing their published list of crawler IPs so you don't accidentally allow someone from a random GCP IP to crawl you if unwanted ( https://developers.google.com/crawling/docs/crawlers-fetcher... ).

Also Googlebot is an AI training crawler so if you block AI training crawlers you should just block it.

Re: Someone is running mass vulnerability scans, spoofing AI bots like ClaudeBot

#175

Post by inigyou »

gavinhking wrote: post #44
dewey wrote: post #40

Earlier quoted context omitted.

You forgot the "Yes, that's my company" part in your reply ( https://ghking.co )

That doesn't change the data/math brother

It changes our evaluation of the likely reasons that you are ignoring the reasons that you are wrong.

Re: Someone is running mass vulnerability scans, spoofing AI bots like ClaudeBot

#176

Post by sgc »

tommica wrote: post #132
cpburns2009 wrote: post #106

Earlier quoted context omitted.

Exactly this. It's no different from a bot pretending to be Googlebot. I've tried reporting abusive IPs to various foreign hosts, but nothing every comes to it. I've settled for just blacklisting excessively abusive IP ranges.

What is your way of detecting them? Just cat your way through your logs?

fail2ban

Re: Someone is running mass vulnerability scans, spoofing AI bots like ClaudeBot

#177

Post by newHempter »

yabones wrote: post #6

Every server with port 80/443 open has thousands of hits a day from random boxes looking for wordpress login pages. The only new thing is that they're pretending to be a different type of annoying bot. There's a new layer of sophistication and subterfuge, but it's the same junk traffic we've always dealt with.

Definitely gotten worse from where I'm standing — you used to be able to just filter on the UA string and move on, now you can't even trust that. Same junk traffic, but harder to sweep away.

Re: Someone is running mass vulnerability scans, spoofing AI bots like ClaudeBot

#179

Post by mcmcmc »

oasisbob wrote: post #145

Very similar experience here. Started July 30, sustained through August 6, when it started a significant ramp-up in volume (5x or so). Most of the traffic is originating in GCP. We're seeing ~70k req/min sustained from Google Cloud IP space (AS396982). Reported to GCP Abuse, they've been non-responsive so far. The main distinguishing factor is the reuse of a bunch of legit AI-training bot UserAgent strings. It's clea…

If how they’ve handled Gmail abuse is any indicator, they’re not likely to do anything. They’re still getting paid for the server time

Re: Someone is running mass vulnerability scans, spoofing AI bots like ClaudeBot

#180

Post by mcmcmc »

Gigachad wrote: post #134
esskay wrote: post #75

Earlier quoted context omitted.

Because businesses dont want them blocked, that would be a very stupid thing for most of them to do given its becoming a vital traffic source now that people are using chatbots instead of google.

From what I have seen at work, everyone is using chat bots but no one is visiting websites through them. We still get almost all traffic through social media and google search.

How many from a direct link without a tracker? That’s usually how I use it for search, get the answer, find the sources and copy the link to a new tab
Post reply on HN