Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ban me at the IP level if you don't like me

boston.conman.org

Post reply on HN
Print view View original

261–270 of 516 posts

Re: Ban me at the IP level if you don't like me

#261

Post by imiric »

seer wrote: post #214
imiric wrote: post #126

Earlier quoted context omitted.

Lately I've been thinking that the only viable long-term solution are allowlists instead of blocklists. The internet has become a hostile place for any public server, and with the advent of ML tools, bots will make up far more than the current ~50% of all traffic. Captchas and bot detection is a losing strategy as bot behavior becomes more human-like. Governments will inevitably enact privacy-infringing regulation to…

I guess this is what "Identity aware proxy" from GCP can do for you? Outsource all of this to google - where you can connect your own identity servers, and then your service will only be accessed after the identity has been verified. We have been using that instead of VPN and it has been incredibly nice and performant.

Yeah, I suppose it's something like that. Except that my solution wouldn't rely on Google, would be open source and self-hostable. Are you aware of a similar project that does this? Would save me some time and effort. :)

There also might be similar solutions for other cloud providers or some Kubernetes-adjacent abomination, but I specifically want something generic and standalone.

Re: Ban me at the IP level if you don't like me

#262

Post by ralferoo »

Out of spite, I'd ignore their request to filter by IP (who knows what their intent is by saying that - maybe they're connecting from VPNs or tor exit nodes to cause disruption etc), but instead filter by matching for that content in the User-Agent instead and feeding them a zip bomb instead.

Re: Ban me at the IP level if you don't like me

#263

Post by 542354234235 »

lxgr wrote: post #45
raffraffraff wrote: post #36

Earlier quoted context omitted.

And across the water, my wife has banned US IP addresses from her online shop once or twice. She runs a small business making products that don't travel well, and would cost a lot to ship to the US. It's a huge country with many people. Answering pointless queries, saying "No, I can't do that" in 50 different ways and eventually dealing with negative reviews from people you've never sold to and possibly even never ta…

As long as your customer base never travels and needs support, sure, I guess. The only way of communicating with such companies are chargebacks through my bank (which always at least has a phone number reachable from abroad), so I’d make sure to account for these.

Usually CC companies require email records (another way of communicating with a company) showing you attempted to resolve the problem but could not. I don’t think “I tried to visit the website that I bought X item from while in Africa and couldn’t get to it” is sufficient.

Re: Ban me at the IP level if you don't like me

#264

Post by ralferoo »

alphazard wrote: post #260

I'm always a little surprised to see how many people take robots.txt seriously on HN. It's nice to see so many folks with good intentions. However, it's obviously not a real solution. It depends on people knowing about it, and adding the complexity of checking it to their crawler. Are there other more serious solutions? It seems like we've heard about "micropayments" and "a big merkle tree of real people" type soluti…

> It depends on people knowing about it, and adding the complexity of checking it to their crawler.

I can't believe any bot writer doesn't know about robots.txt. They're just so self-obsessed and can't comprehend why the rules should apply to them, because obviously their project is special and it's just everyone else's bot that causes trouble.

Re: Ban me at the IP level if you don't like me

#265

Post by ralferoo »

phito wrote: post #134
bob1029 wrote: post #132

I think a lot of really smart people are letting themselves get taken for a ride by the web scraping thing. Unless the bot activity is legitimately hammering your site and causing issues (not saying this isn't happening in some cases), then this mostly amounts to an ideological game of capture the flag. The difference being that you'll never find their flag. The only thing you win by playing is lost time. The best wa…

My friend has a small public gitea instance, only use by him a a few friends. He's getting thousounds of requests an hour from bots. I'm sorry but even if it does not impact his service, at the very least it feels like harassment

What's worse is when you get bots blasting HTTP traffic at every open port, even well known services like SMTP. Seriously, it's a mail server. It identified itself as soon as the connection was opened, if they waited 100ms-300ms before spamming, they'd know that it wasn't HTTP because the other side wouldn't send anything at all if it was. There's literally no need to bombard a mail server on a well known port by continuing to send a load of junk that's just going to fill someone's log file.

Re: Ban me at the IP level if you don't like me

#266

Post by flanbiscuit »

breve wrote: post #223

> Alex Schroeder's Butlerian Jihad That's Frank Herbert's Butlerian Jihad.

To be fair, he was referring to a post on Alex Schroeder's blog titled with the same name as the term from the Dune books. And that post correctly credits Dune/Herbert. But the post is not about Dune, it's about Spam bots so it's more related to what the original author's post is about.

Speaking of the Butlerian Jihad, Frank Herbert's son (Brian) and another author named Kevin J Anderson co-wrote a few books in the Dune universe and one of them was about the Butlerian Jihad. I read it. It was good, not as good at Frank Herbert's books but I still enjoyed it. One of the authors is not as good as the other because you can kind of tell the writing quality changing per chapter.

https://en.wikipedia.org/wiki/Dune:_The_Butlerian_Jihad

Re: Ban me at the IP level if you don't like me

#267

Post by Bender »

johnisgood wrote: post #123
sugarpimpdorsey wrote: post #38

Earlier quoted context omitted.

There's some weird ones you'd never think of that originate an inordinate amount of bad traffic. Like Seychelles. A tiny little island nation in the middle of the ocean inhabited by... bots apparently? Cyprus is another one. Re: China, their cloud services seem to stretch to Singapore and beyond. I had to blacklist all of Alibaba Cloud and Tencent and the ASNs stretched well beyond PRC borders.

[flagged]

Ignore the trolls. Also, if they are upset with you they should focus their vitriol on me. I block nearly all of BRICS especially Brazil as most are hard wired to not follow even the simplest of rules, most data-centers, some VPN's based on MSS, posting from cell phones and much more. I am always happy to give people the satisfaction of down-voting me since I use uBlock to hide karma.

In ublock -> my filters

    # HN Block Karma View
    news.ycombinator.com##.comhead .score:style(overflow: hidden; display: inline-block; line-height: 0.1em; width: 0; margin-left: -1.9em;)
    news.ycombinator.com###hnmain > tbody > tr:first-of-type table td:last-of-type .pagetop:style(font-size: 0!important; color: transparent!important;)
    news.ycombinator.com###hnmain > tbody > tr:first-of-type table td:last-of-type .pagetop > *:style(font-size: 10pt; line-height: 1.45em;)
    news.ycombinator.com###logout::before:style(content: "|"; padding: 0.25em;)
    news.ycombinator.com##form.profileform tbody tr:nth-child(3)

Re: Ban me at the IP level if you don't like me

#268

Post by Bender »

azthecx wrote: post #160
Xiol32 wrote: post #144

Earlier quoted context omitted.

Changing the SSH port also helps cut down the noise, as part of a layered strategy.

Did you really notice a significant drop off in connection attempts? I tried this some years ago and after a few hours on a random very high port number I was already seeing connections.

I use a non standard port and have not had an unknown IP hit it in over 25 years. It's not a security feature for me, I use that to avoid noise.

My public SFTP servers are still on port 22 and but block a lot of SSH bots by giving them a long "versionaddendum" /etc/ssh/sshd_config as most of them choke on it. Mine is 720 characters long. Older SSH clients also choke on this so test it first if going this route. Some botters will go out of their way to block me instead so their bots don't hang. One will still see the bots in their logs, but there will be far less messages and far fewer attempts to log in as they will be broken, sticky and confused. Be sure to add offensive words in versionaddendum for the sites that log SSH banners and display them on their web pages like shodan.io.

Re: Ban me at the IP level if you don't like me

#269

Post by Bender »

ralferoo wrote: post #264
alphazard wrote: post #260

I'm always a little surprised to see how many people take robots.txt seriously on HN. It's nice to see so many folks with good intentions. However, it's obviously not a real solution. It depends on people knowing about it, and adding the complexity of checking it to their crawler. Are there other more serious solutions? It seems like we've heard about "micropayments" and "a big merkle tree of real people" type soluti…

> It depends on people knowing about it, and adding the complexity of checking it to their crawler. I can't believe any bot writer doesn't know about robots.txt. They're just so self-obsessed and can't comprehend why the rules should apply to them, because obviously their project is special and it's just everyone else's bot that causes trouble.

(malicious) Bot writers have exactly zero concern for robots.txt. Most bots are malicious. Most bots don't set most of the TCP/IP flags. Their only concern is speed. I block about 99% of port scanning bots by simply dropping any TCP SYN packet that is missing MSS or uses a strange value. The most popular port scanning tool is masscan which does not set MSS and some of the malicious user-agents also set some odd MSS values if they even set it at all.

    -A PREROUTING -i eth0 -p tcp -m tcp -d $INTERNET_IP --syn -m tcpmss ! --mss 1280:1460 -j DROP
Example rule from the netfilter raw table. This will not help against headless chrome.

The reason this is useful is that many bots first scan for port 443 then try to enumerate it. The bots that look up domain names to scan will still try and many of those come from new certs being created in LetsEncrypt. That is one of the reasons I use the DNS method, get a wildcard and sit on it for a while.

Another thing that helps is setting a default host in ones load balancer or web server that serves up a default simple static page served from a ram disk that say something like, "It Worked!" and disable logging for that default site. In HAProxy one should look up the option "strict-sni". Very old API clients can get blocked if they do not support SNI but along that line most bots are really old unsupported code that the botter could not update if their life depended on it.

Re: Ban me at the IP level if you don't like me

#270

Post by maldonad0 »

zelphirkalt wrote: post #206
phplovesong wrote: post #53

We block China and Russia. DDOS attacks and other hack attempts went down by 95%. We have no chinese users/customers so in theory this does not effect business at all. Also russia is sanctioned and our russian userbase does not actually live in russia, so blocking russia did not effect users at all.

Which will ensure you never get customers from those countries. And so the circle closes ...

I used to run a forum that blocked IPs from outside Western Europe. I had no interest in users from beyond. It's not all about money.
Post reply on HN