Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Someone is running mass vulnerability scans, spoofing AI bots like ClaudeBot

knownagents.com

Post reply on HN
Print view View original

221–230 of 233 posts

Re: Someone is running mass vulnerability scans, spoofing AI bots like ClaudeBot

#221

Post by Jskewel »

an0malous wrote: post #101
drewnick wrote: post #21

Earlier quoted context omitted.

Think about how many webmaster and business owners' egos are stroked by all the traffic they are getting, when in actuality they are often just serving thousands of bots.

Is that basically the delta between Cloudflare’s Pages analytics and Google Analytics? CF says I get thousands of visits a day and GA says it’s like 3-4 users.

Cloudflare analytics is great, but you need to filter by edge status code 200 to see the actual real traffic that was allowed through their firewall.

Re: Someone is running mass vulnerability scans, spoofing AI bots like ClaudeBot

#222

Post by [deleted] »

tommica wrote: post #132
cpburns2009 wrote: post #106

Earlier quoted context omitted.

Exactly this. It's no different from a bot pretending to be Googlebot. I've tried reporting abusive IPs to various foreign hosts, but nothing every comes to it. I've settled for just blacklisting excessively abusive IP ranges.

What is your way of detecting them? Just cat your way through your logs?

[deleted]

Re: Someone is running mass vulnerability scans, spoofing AI bots like ClaudeBot

#225

Post by 01284a7e »

jasonjayr wrote: post #68
0xdeadbeefbabe wrote: post #46

Earlier quoted context omitted.

I don't even have a domain name and I'm getting lots of hits.

I recently brought up a website on a never-before-seen .com domain. Within about 10 mins of bringing it up with a SSL certificate, Anthropic came knocking on the door requesting the front page. (Almost certainty due to them watching the Public Certificate Transparency logs)

"Public Certificate Transparency logs"...

The scam of "everyone should have SSL" right here, ladies and gentlemen.

Re: Someone is running mass vulnerability scans, spoofing AI bots like ClaudeBot

#226

Post by Jskewel »

inigyou wrote: post #166
random29ah wrote: post #81

Earlier quoted context omitted.

Getting it directly from IANA is always the best approach. Here is a "simplified" version in various formats. https://github.com/HotCakeX/Official-IANA-IP-blocks

Gross - you're going to block countries just because the US government doesn't like those countries?

I'm happy to unblock China from accessing my website, once China does something about the millions of daily hack attempts originating from that country.

Re: Someone is running mass vulnerability scans, spoofing AI bots like ClaudeBot

#227

Post by gregoriol »

Jskewel wrote: post #223

For those suggesting fail2ban as a solution, that's dinosaur software from the palaeolithic. If you have a website of any size then the number of bots will overwhelm the block list in days with their millions of unique IPs.

Unique IPs is not something AI can generate easily, and blocking IPs in general is not efficient.

The big problem with AIs is that they can try new paths/payloads very easily, adapt quickly, I'd be more worried about that.

Re: Someone is running mass vulnerability scans, spoofing AI bots like ClaudeBot

#228

Post by fragmede »

unclebucknasty wrote: post #217
b112 wrote: post #212

Earlier quoted context omitted.

Re-read the post. He converted the sites to static html.

To GP's point, any connected server can become a liability, even if it is intended to just host static HTML (or for some other purpose).

I'm serving static sites via Cloudflare pages. Pretty sure that's gonna be actually static.

Re: Someone is running mass vulnerability scans, spoofing AI bots like ClaudeBot

#229

Post by unclebucknasty »

fragmede wrote: post #228
unclebucknasty wrote: post #217

Earlier quoted context omitted.

To GP's point, any connected server can become a liability, even if it is intended to just host static HTML (or for some other purpose).

I'm serving static sites via Cloudflare pages. Pretty sure that's gonna be actually static.

You're running Cloudflare Pages on a home server?

Re: Someone is running mass vulnerability scans, spoofing AI bots like ClaudeBot

#230

Post by Jskewel »

ChillyCapy wrote: post #87

Fake Googlebot visits are #1 in website logs I've been working on. At the beginning I was fighting with them using Cloudflare ASN block rules or their managed Bot Fight mode but it appeared to be not only pointless, but also harmful for my websites. Bot Fight mode randomly started blocking real Bing / Google / OpenAI crawlers what wasted crawling budget and discouraged crawlers to revisit updated pages. Sometimes it'…

With Cloudflare you can set a rule to block traffic that identifies as Googlebot but is not a "verified bot", ie is not from the proper IP range.
Post reply on HN