Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Botspam apocalypse

memex.marginalia.nu

Post reply on HN
Print view View original

181–190 of 358 posts

Re: Botspam apocalypse

#181

Post by nicbou »

nixcraft wrote: post #95

I run a popular blog and confirm that spam is a massive issue. I am trying to keep the independent web alive with an old-school commenting system because it helps readers and myself improve outdated posts. My domain is over 20+ years old and attracts all sorts of threats, including monthly DDoS and daily spam. Using Cloudflare solved all of these problems. Next, you need to add firewall rules inside Cloudflare WAF to…

Same job, same problem. I simply don't allow comments anymore.

This is unfortunate, because they're amazing feedback if you write about bureaucracy. People won't take the time to write to you about their experience, but they'll leave a comment.

Re: Botspam apocalypse

#182

Post by 1vuio0pswjnm7 »

Avamander wrote: post #108

It's annoying for sure. I deal with abuse at a large scale. I'd recommend: - Rate-limit everything, absolutely everything. Set sane limits. - Rate-limit POST requests harder. Preferably dynamically based on geoip. - Rate-limit login and comment POST requests even harder. Ban IPs that exceed the amount. - Require TLS. Drop TLSv1.0 and TLSv1.1. Bots certainly break. - Require SNI. Do not reply without SNI (nginx has 44…

"This spam traffic is all from botnets with IPs all over the world. Tens, maybe hundreds of thousands of IPs, each with a relatively modest query rates, so rate limiting does all of bupkis."

Re: Botspam apocalypse

#183

Post by shp0ngle »

The actual blogpost aside: the margianalia search is the first of these “alternative search engines” that I actually like.

Most of those has been either “worse google” or “utter trash”… this one returns some interesting results for some queries I have tried.

Re: Botspam apocalypse

#184

Post by Avamander »

troad wrote: post #166
Avamander wrote: post #108

It's annoying for sure. I deal with abuse at a large scale. I'd recommend: - Rate-limit everything, absolutely everything. Set sane limits. - Rate-limit POST requests harder. Preferably dynamically based on geoip. - Rate-limit login and comment POST requests even harder. Ban IPs that exceed the amount. - Require TLS. Drop TLSv1.0 and TLSv1.1. Bots certainly break. - Require SNI. Do not reply without SNI (nginx has 44…

In other words, make your website unusable for people who have to connect through VPNs or public networks, difficult for anyone without a stable Western broadband connection, and unpleasant for everyone else.

With a bit of work any limits can be fine-tuned usually not to impact actual users behind NATs. Some collateral does happen but that's an unfortunate reality. I'd like you to elaborate on the rest of your comment though.

Re: Botspam apocalypse

#185

Post by protoduction »

tmikaeld wrote: post #179
protoduction wrote: post #163

Earlier quoted context omitted.

I’m the co-founder of Friendly Captcha [0], we offer a proof of work-based captcha since two years or so. Happy to answer any questions. A big part of what makes our captcha successful in fighting abuse is that we scale the difficulty of the proof-of-work puzzle based on the user’s previous behavior and other signals (e.g. minus points if their IP address is a known datacenter IP). The nice thing about a scaling PoW…

While your service is of high quality, the pricing is completely unreasonable for private use cases, many times higher than hosting the site in the first place.

I'm sorry to hear that. We offer free and small plans for small use-cases, but I also understand that some projects don't have a budget at all.

There is a blessed source-available version of the server that you can self-host [0]. It is more limited in its protection, but it is probably good enough for hobby projects.

[0]: https://github.com/FriendlyCaptcha/friendly-lite-server

Re: Botspam apocalypse

#186

Post by Avamander »

1vuio0pswjnm7 wrote: post #182
Avamander wrote: post #108

It's annoying for sure. I deal with abuse at a large scale. I'd recommend: - Rate-limit everything, absolutely everything. Set sane limits. - Rate-limit POST requests harder. Preferably dynamically based on geoip. - Rate-limit login and comment POST requests even harder. Ban IPs that exceed the amount. - Require TLS. Drop TLSv1.0 and TLSv1.1. Bots certainly break. - Require SNI. Do not reply without SNI (nginx has 44…

"This spam traffic is all from botnets with IPs all over the world. Tens, maybe hundreds of thousands of IPs, each with a relatively modest query rates, so rate limiting does all of bupkis."

Yep, there isn't a silver bullet that curtails all abuse.

Re: Botspam apocalypse

#187

Post by nicbou »

hamilyon2 wrote: post #5

I experienced this firsthand with government immigration websites. The thing is there are only so many time slots and and people are forsed to use a certain web site to apply, so everyone is hunting for available time and generally none are available. So, some creative people set up bots which check periodically for them. They are paid services which will do that for you. Now we have bots hammering gatekeeper's websi…

Berlin?

Re: Botspam apocalypse

#188

Post by AviationAtom »

It's funny the author mentions Facebook and Twitter, because the bot spam on both is quite apparent. The spam on the former has risen greatly, seemingly mostly from India and parts of Africa. Scam air duct cleaning posts, posts about hacked account recovery, and random other crap. It really degrades the experience of the Internet, IMHO.

Re: Botspam apocalypse

#189

Post by Aachen »

Animats wrote: post #39

What's hard to do now is host a lightly used but broadly interesting service that doesn't require a login. Although, surprisingly, I host such a service, and while it gets a constant stream of random hits, they're a minor nuisance. Probably because it's just the back end for a web page, and nobody bothers to target it specifically. Random web browsing won't find it, and the API will just return an error if called inc…

Same! I also got like 20 requests every second from a university IP. I tried a few things to make it error out, like returning 404, but no dice. In my case, it was my own fault though, a page with a few lines of JS to periodically check for updates got into a crazy state (I never found out how) and they didn't notice because it was a remote desktop system where they left the page open. Went on for months but didn't impact my service (I just noticed it in access logs while looking for something else) so I left it and remembered again a few months later, then it was gone.

Re: Botspam apocalypse

#190

Post by marginalia_nu »

ajsnigrutin wrote: post #173
Avamander wrote: post #108

It's annoying for sure. I deal with abuse at a large scale. I'd recommend: - Rate-limit everything, absolutely everything. Set sane limits. - Rate-limit POST requests harder. Preferably dynamically based on geoip. - Rate-limit login and comment POST requests even harder. Ban IPs that exceed the amount. - Require TLS. Drop TLSv1.0 and TLSv1.1. Bots certainly break. - Require SNI. Do not reply without SNI (nginx has 44…

> - Rate-limit everything, absolutely everything. Set sane limits. This breaks when multiple users are behind the same IP. I've seen services fail even in classroom, because the prof did something and a few tens of students followed (captchas everywhere).

(Author)

I do in fact rate-limit everything, it is good advice, but the way you implement rate-limiting allows for traffic bursts. It's basically a reverse leaky bucket, where you start out with N allowed requests, which gets depleted for each request, and refilled slowly over time.

Search traffic is fairly bursty, people do a few requests where they tweak the query and then they go go away.

Post reply on HN