Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Someone is running mass vulnerability scans, spoofing AI bots like ClaudeBot

knownagents.com

Post reply on HN
Print view View original

131–140 of 242 posts

Re: Someone is running mass vulnerability scans, spoofing AI bots like ClaudeBot

#131

Post by Bender »

jwally wrote: post #127
Bender wrote: post #122

Earlier quoted context omitted.

Not the person you are asking but site operators can not tell intent. It could be something nasty on the network or a botter feigning ignorance. I'd say its probably an acceptable casualty in the battleground that is the internet especially for little one-off sites hosting blogs, forums, chat servers, etc... For a bigger site I would expect that person may have to open a ticket with the platform such as Amazon accept…

thats the hard part, right - my 76 year old dad is on his banking app while his samsung TV is allowing a bot to try and take over other accounts at the same bank on the same IP. IP Blacklists, no matter how good can't stop this. You have to start using stats or deep-diving telemetry. https://darknetdiaries.com/episode/172/

thats the hard part, right - my 76 year old dad is on his banking app while his samsung TV is allowing a bot to try and take over other accounts at the same bank on the same IP.

So appeal to emotion doesn't fly with me. If grandpa is 76 in the year of our lord 2026 that means he was 50 when the internet was getting popular and 59 when cell phones became very popular on the internet. He's not much older than I. He knows what's up.

God help the makers of that television if he finds out it has been spying on him and dorking around with his traffic. If they are lucky he will just take a baseball bat to it. If they are unlucky he will fly to their headquarters and end up on a viral bodycam video likely with a lot of supporters that will bail him out of jail.

IP Blacklists, no matter how good can't stop this. You have to start using stats or deep-diving telemetry.

I use a myriad of methods including IP blacklists. That's my choice and every site operators choice. I do not have to use deep-diving telemetry but you are free to do so.

Re: Someone is running mass vulnerability scans, spoofing AI bots like ClaudeBot

#132

Post by tommica »

cpburns2009 wrote: post #106
yabones wrote: post #6

Every server with port 80/443 open has thousands of hits a day from random boxes looking for wordpress login pages. The only new thing is that they're pretending to be a different type of annoying bot. There's a new layer of sophistication and subterfuge, but it's the same junk traffic we've always dealt with.

Exactly this. It's no different from a bot pretending to be Googlebot. I've tried reporting abusive IPs to various foreign hosts, but nothing every comes to it. I've settled for just blacklisting excessively abusive IP ranges.

What is your way of detecting them? Just cat your way through your logs?

Re: Someone is running mass vulnerability scans, spoofing AI bots like ClaudeBot

#134

Post by Gigachad »

esskay wrote: post #75
Tharre wrote: post #51

Why would you voluntarily pretend to be a AI bot, when those have already a much higher chance of being blocked? Seems holly unproductive. Best hypothesis I can come up with is to somehow make the AI companies look bad, but they seem to be doing an excellent job at that themselves already by scraping everyone hundreds of times per hour over and over.

Because businesses dont want them blocked, that would be a very stupid thing for most of them to do given its becoming a vital traffic source now that people are using chatbots instead of google.

From what I have seen at work, everyone is using chat bots but no one is visiting websites through them. We still get almost all traffic through social media and google search.

Re: Someone is running mass vulnerability scans, spoofing AI bots like ClaudeBot

#136

Post by numpad0 »

thesuitonym wrote: post #55
tommyage wrote: post #45

Earlier quoted context omitted.

I, temporarly, banned some ip range. I didn't find a source for pinpointing countries; though I am interested. Could you point me to some sources which, deterministically, resolve to some countries? To my knowledge you can not reliably identify countries by ip since this would be dependent on DNS servers. Though I am just a application programmer! Thanks in advance.

Your router doesn't care about their DNS settings. IP addresses are very easy to tie back to countries. The reason they say it's not reliable is because it's trivial to spoof the country, but even so, a lot of attackers don't even bother. It's sort of like the Nigerian prince scam calls: if you're wise enough to block Russia, you're not worth their time. Your firewall vendor should supply you with country lists, just…

> The reason they say it's not reliable is because it's trivial to spoof the country

ISPs sometimes do trade IPv4 blocks and countries to which it belongs do change occasionally. That can become a problem if you were like literally Netflix and someone few nation states over started an ISP.

Re: Someone is running mass vulnerability scans, spoofing AI bots like ClaudeBot

#137

Post by 0xdeadbeefbabe »

xplt wrote: post #66
0xdeadbeefbabe wrote: post #46

Earlier quoted context omitted.

I don't even have a domain name and I'm getting lots of hits.

Old and busted: serverless New hotness: DNS-less

Yeah it's even superior to smol web in some circles. It's not that much longer than a phone number.

Re: Someone is running mass vulnerability scans, spoofing AI bots like ClaudeBot

#138

Post by dylan604 »

iwontberude wrote: post #70
manapause wrote: post #62

Earlier quoted context omitted.

This is so true. Every junior sysadmin I have trained over the years (including myself) has had a “are we being attacked?!” moment when tasked with WAF report analysis, monitoring fail2ban logs, etc. Monitoring WAN traffic really gets the paranoia juices flowing.

I remember when you could stand up a website and no bots would scrape it or scan it. It was a lovely time. No one had firewalls or antivirus and things were working fine until the worms and viruses started coming. You could be confident that your guests were real, so much so we had guest counters on many public sites.

Remember when you had to submit a request for google to scan your site?

Re: Someone is running mass vulnerability scans, spoofing AI bots like ClaudeBot

#139

Post by cullenking »

VladVladikoff wrote: post #14
codegeek wrote: post #12

Earlier quoted context omitted.

Is there an easy way to block any requests originating from VPS etc instead of residential/commercial IP from legitimate users ? I know cloudflare does a few things but I really want to figure out a way to block any request say at nginx or caddy (reverse proxy) from reaching origin servers if they are not from an IP that is not a VPS etc.

Yes but it’s not cheap. Maxmind and ipinfo etc sell a tier that tells you this information, then you can 403 based on it. But the price is nuts like $40,000 a year.

See my comment in the parent - there are cheaper options, don't use maxmind!

Re: Someone is running mass vulnerability scans, spoofing AI bots like ClaudeBot

#140

Post by cullenking »

Bender wrote: post #2

Many of those user-agents listed are often faked. Look up which ASN owns their IP. If I block most VPS providers most of the faked bots vanish. There are still some running from residential and phones using hijacked code (readers that are not really just readers but really multipurpose proxies) . On that note, do not trust the linked source code but rather decompile the live code your phone is running and have AI ana…

I did just this. Using a $2k a year database from a smaller provider that isn't maxmind, claude and I built a pretty slick ASN based categorization system. I can categorize an ASN as a residential IP, a service provider, a legit crawler/scraper, etc. For anything that is suspicious, I dynamically use turnstile to gate access to our service. Turns out there's no ISP for any VPN, they just contract with a shitload of mom and pop shady colocation services across the world.

We collect signals that help determine good vs bad networks. For example, large amounts of requests to .php endpoints, large amounts of empty accounts from the same /24 subnet, etc etc. All these signals let us automatically determine risk, and then put up a challenge. Authenticated users never see the challenge even if they are on a risky network (VPN 99.9% of the time), unless the network has been identified as 100% malicious, then it gets a full block.

Here's a small snapshot of the dashboard:

https://cos.ridewithgps.com/screenshots/6a7c54d0-12Aug26-358...

This was probably a total of 3-4 days of work, spread out over a couple months of iterative claude led hacking. I didn't know exactly what to build, but had some of the key architectural ideas in my head. Opus+Faable made easy work of it all, and ended up guiding some really slick improvements for performance.

I would say this has dropped about 20% of all traffic to our service, though it turns out turnstile is a massive target for bots, so replacing that with something custom is next on the list.

Post reply on HN