Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ban me at the IP level if you don't like me

boston.conman.org

Post reply on HN
Print view View original

191–200 of 516 posts

Re: Ban me at the IP level if you don't like me

#191

Post by sim7c00 »

poisonborz wrote: post #187

Naive question: why isn't there a publicly accessible central repository of bad IPs and domains, stewarded by the industry, operated by a nonprofit, like W3C? Yes it wouldn't be enough by itself ("bad" is a very subjective term) but it could be a popular well-maintained baseline.

there are many of these and they are always outdated

another issue is things like cloud hosting will overlap their ranges with legit business ranges happily, so if you go that route you will inadvertently also block legitimate things. not that a regular person care too much for that, but an abuse list should be accurate.

Re: Ban me at the IP level if you don't like me

#193

Post by sim7c00 »

dmurray wrote: post #77
firefoxd wrote: post #26

Since I posted an article here about using zip bombs [0], I'm flooded with bots. I'm constantly monitoring and tweaking my abuse detector, but this particular bot mentioned in the article seemed to be pointing to an RSS reader. I white listed it at first. But now that I gave it a second look, it's one of the most rampant bot on my blog. [0]: https://news.ycombinator.com/item?id=43826798

If I had a shady web crawling bot and I implemented a feature for it to avoid zip bombs, I would probably also test it by aggressively crawling a site that is known to protect itself with hand-made zip bombs.

also protect yourself fromnsucking up fake generated content. i know some folks here like to feed them all sorts of 'data' . fun stuff :D

Re: Ban me at the IP level if you don't like me

#194

Post by nubinetwork »

roguebloodrage wrote: post #64

This is everything I have for AS132203 (Tencent). It has your addresses plus others I have found and confirmed using ipinfo.io 43.131.0.0/18 43.129.32.0/20 101.32.0.0/20 101.32.102.0/23 101.32.104.0/21 101.32.112.0/23 101.32.112.0/24 101.32.114.0/23 101.32.116.0/23 101.32.118.0/23 101.32.120.0/23 101.32.122.0/23 101.32.124.0/23 101.32.126.0/23 101.32.128.0/23 101.32.130.0/23 101.32.13.0/24 101.32.132.0/22 101.32.132.…

FWIW, I looked through my list of ~8000 IP addresses, there isn't as many hits for these ranges as I would have thought. It's possible that they're more focused on using known DNS names than simply connecting to 80/443 on random IPs.

Edit: I also checked my Apache logs, I couldn't find any recent logs for "thinkbot".

Re: Ban me at the IP level if you don't like me

#195

Post by calgoo »

motorest wrote: post #183
lxgr wrote: post #31

Earlier quoted context omitted.

Why stop there? Just block all non-US IPs! If it works for my health insurance company, essentially all streaming services (including not even being able to cancel service from abroad), and many banks, it’ll work for you as well. Surely bad actors wouldn’t use VPNs or botnets, and your customers never travel abroad?

> Why stop there? Just block all non-US IPs! This is a perfectly good solution to many problems, if you are absolutely certain there is no conceivable way your service will be used from some regions. > Surely bad actors wouldn’t use VPNs or botnets, and your customers never travel abroad? Not a problem. Bad actors which are motivated enough to use VPNd or botnets are a different class of attacks that have different t…

This. If someone wants to target you, they will target you. What this does is remove the noise and 90%+ of crap.

Basically the same thing as changing the ssh port on a public facing server, reduce the automated crap attacks.

Re: Ban me at the IP level if you don't like me

#197

Post by sim7c00 »

speleding wrote: post #172
roguebloodrage wrote: post #64

This is everything I have for AS132203 (Tencent). It has your addresses plus others I have found and confirmed using ipinfo.io 43.131.0.0/18 43.129.32.0/20 101.32.0.0/20 101.32.102.0/23 101.32.104.0/21 101.32.112.0/23 101.32.112.0/24 101.32.114.0/23 101.32.116.0/23 101.32.118.0/23 101.32.120.0/23 101.32.122.0/23 101.32.124.0/23 101.32.126.0/23 101.32.128.0/23 101.32.130.0/23 101.32.13.0/24 101.32.132.0/22 101.32.132.…

For the Thinkbot problem mentioned in the article, it's less maintenance work to simply block on the User Agent string.

jep, good tip! for ppl that do this be sure to make it case insensitive and only capture few distinct parts, not too specific. especially if u only expect browsers this can mitigate a lot.

u can also filter for allowing but this gives a risk of allowing the wrong thing as headers are easy to set, so its better to do it via blocking (sadly)

Re: Ban me at the IP level if you don't like me

#198

Post by sim7c00 »

i think there is an opportunity to train an neural network on browser user agent s(they are catalogued but vary and change a lot). then u can block everything not matching.

it will work better than regex. a lot of these companies rely on 'but we are clearly recognizable' via fornexample these user agents, as excuse to put burden on sysadmins to maintains blocklists instead of otherway round (keep list of scrapables..)

maybe someone mathy can unburden them ?

you could also look who ask for nonexisting resources, and block anyone who asks for more than X (large enough not to let config issue or so kill regular clients). block might be just a minute so u dont have too many risk when an FP occurs. it will be enough likely to make the scraper turn away.

there are many things to do depending on context, app complexity, load etc. , problem is there's no really easy way to do these things.

ML should be able to help a lot in such a space??

Re: Ban me at the IP level if you don't like me

#199

Post by throwaway290 »

plaguna wrote: post #14
dokyun wrote: post #11

Earlier quoted context omitted.

Who cares?

Same people who care about “master” and “main” for hit branches.

Master branch comes from Latin for expert, authority. master record also comes from that meaning.

Blacklist and whitelist come from black=bad and white=good which if you are black or have empathy is a red flag

Re: Ban me at the IP level if you don't like me

#200

Post by nailer »

mnw21cam wrote: post #192
lmz wrote: post #179

Earlier quoted context omitted.

> be a hero and die a martyr I believe it's "an hero".

Uh, no, it's definitely not. Hero begins with a consonant, so it should be preceded by "a", not "an".

Welcome to British English. The h in hero isn’t pronounced, same as hospital, so you use an before it.
Post reply on HN