Live data from Hacker News

Botspam apocalypse

memex.marginalia.nu

211–220 of 358 posts

Re: Botspam apocalypse

#211

So there’s one service keeping this search engine online, and it’s probably doing it for free, and the author can’t even think of a better way to do it. Yet Cloudflare still gets two paragraphs of complaints in the face? Because the author wants to “own” something instead of “renting”?

I'm doing it for free because I don't want this to be a commercial service. I get that HN is startup city, but I'm not running a startup, it's just a hobby.

Re: Botspam apocalypse

#212
post #158

Earlier quoted context omitted.

Genuinely curious, why disable HTTP2? Your web browsing must be awfully slow sans multiplexing.

> why disable HTTP2 Because it adds nothing to improve my browsing experience, and reducing the number of protocols supported by my browser from 3 to 1 also reduces the attack surface. > Your web browsing must be awfully slow sans multiplexing. And yet it's not slowed down at all. How many different resources must a web page use before it feels slow on a connection pool of keep-alive TCP sockets? Maybe people visit s…

Perhaps your experience is the same, but it may impose extra load on middleboxes that track TCP flows.

Re: Botspam apocalypse

#213
post #111

Earlier quoted context omitted.

How does that work without becoming a SPOF for taking down the website ? Can't a user/botnet with more CPU power than the server simply send more captchas than can be processed ? In addition, using sha256 for this is IMHO a mistake, calling for ASIC abuse.

> How does that work without becoming a SPOF for taking down the website ? Can't a user/botnet with more CPU power than the server simply send more captchas than can be processed ? Glad you asked! This is theoretically possible, but the adversary will have to be highly motivated with considerable resources to choke mCaptcha. For instance, to generate Proof of Work(PoW), the client will have to generate 50k hashes(can…

> the client will have to generate 50k hashes(can be configured for higher difficulty)

I completely forgot how PoW worked, it's clearer now. You should probably add that this is a probabilistic average, so people will have to be ready for much longer (and faster) resolutions.

With what you said, an adversary can probably just DoS mCaptcha without any computation, if verification is stateless (by sending garbage at line rate); if it is stateful (e.g CSRF token), you'll have to do a cache query, which is probably on the same order of magnitude of a single hash.

Re: Botspam apocalypse

#214
post #45
post #2

> They're a major part in killing off web forums, and a significant wet blanket on any sort of fun internet creativity or experimentation. > The only ones that can survive the robot apocalypse is large web services. Your reddits, and facebooks, and twitters, and SaaS-comment fields, and discords. They have the economies of scale to develop viable countermeasures, to hire teams of people to work on the problem full ti…

> it costs 10 bux to register an account, if you're a nuisance your account is banned and you pay 10bux to get back on. You've highlighted its biggest tradeoff which is that it creates an economic incentive to ban people. The only way to make more money, is to have more rules and culture for ostracizing people. It would have been smarter of Something Awful (since that's the site we're talking about) to charge $4/mont…

You used to have to really, really try to get banned or even probated on SA, now it doesn't take much at all.

Re: Botspam apocalypse

#215

In the 1980's, we kept anklebyters off dial-up BBSses with a simple technique: voice validation. To join the forum, you had to fill an application first, which included your real name and phone number. The sysop would give you a call for a quick chat, and then grant you access if you didn't seem like a twit. This would be entirely practical for some small-time operator trying to run a forum off residential broadband,…

I remember some BBS registration forms where you would have to give the names of a couple existing users that would vouch for you. Kind of like other sites where you need an invite or referral from an existing member.

Re: Botspam apocalypse

#216
post #147

It's annoying for sure. I deal with abuse at a large scale. I'd recommend: - Rate-limit everything, absolutely everything. Set sane limits. - Rate-limit POST requests harder. Preferably dynamically based on geoip. - Rate-limit login and comment POST requests even harder. Ban IPs that exceed the amount. - Require TLS. Drop TLSv1.0 and TLSv1.1. Bots certainly break. - Require SNI. Do not reply without SNI (nginx has 44…

> If you can, require HTTP/2.0. Bots break. Non-bots break as well. I have Firefox configured to use HTTP/1.1 only. No reason to chase Google's standard-of-the-day, HTTP/1.1 has worked for ages and it will continue to do so for the foreseeable future.

If doing something fends off a lot of bots, but also inconveniences a very small number of people who have significantly non-standard or just out-of-date configurations, I'm likely to favour protecting myself from the former over worrying about the latter. To paraphrase Mr Spok: The inconveniences of the me outweigh the inconveniences of the you!

Re: Botspam apocalypse

#217
post #45
post #2

> They're a major part in killing off web forums, and a significant wet blanket on any sort of fun internet creativity or experimentation. > The only ones that can survive the robot apocalypse is large web services. Your reddits, and facebooks, and twitters, and SaaS-comment fields, and discords. They have the economies of scale to develop viable countermeasures, to hire teams of people to work on the problem full ti…

> it costs 10 bux to register an account, if you're a nuisance your account is banned and you pay 10bux to get back on. You've highlighted its biggest tradeoff which is that it creates an economic incentive to ban people. The only way to make more money, is to have more rules and culture for ostracizing people. It would have been smarter of Something Awful (since that's the site we're talking about) to charge $4/mont…

Nah, there are other ways to raise money. The forums have you pay for avatar changes (or to change others' avatars! which is a fun chunk of the forums culture), there are various upgrades like no-ads and unlocking private messaging and stuff. And the forums now have a Patreon, too.

Re: Botspam apocalypse

#218
post #9

> has been upwards of 15 queries per second from bots What type of queries are they generating? For what purpose are querying Marginalia? Scraping and filling internal search engines? > If anyone could go ahead and find a solution to this mess I would maybe trying to investigate why are querying your search engine. Is for the search results? Maybe from there you can create and sell an API service. Is for the wiki? Is…

Most of the queries don't seem to be tailored toward my search engine, they're ridiculously over-specified and typically don't return any results at all.

As I've mentioned in another comment, my best guess is they're betting it's backed by google, and are attempting to poison their search term suggestions. The queries I've been getting are fairly long and highly specific, often within e-pharma or online casino or similarly sketchy areas.

Like

> cialis 50mg online pharmacy canada price

Either that, or nonsense like the below, where they appear to be looking for CMSes to exploit (although I don't understand the appendage at the end)

> "Please enter the email address associated with your User account. Your username will be emailed to the email address on file." Finestre Antirumore Torino

> affordable local seo services "Din epostadress delas eller publiceras aldrig Obligatoriska flt r markerade med"

> "You are not logged in. (Login)" Country "City/Town" "Web page" erst

Point is, none of these queries actually return anything at all. I don't offer real full text search, for one. And the queries are much too long.

Re: Botspam apocalypse

#219
post #147

Earlier quoted context omitted.

> If you can, require HTTP/2.0. Bots break. Non-bots break as well. I have Firefox configured to use HTTP/1.1 only. No reason to chase Google's standard-of-the-day, HTTP/1.1 has worked for ages and it will continue to do so for the foreseeable future.

If doing something fends off a lot of bots, but also inconveniences a very small number of people who have significantly non-standard or just out-of-date configurations, I'm likely to favour protecting myself from the former over worrying about the latter. To paraphrase Mr Spok: The inconveniences of the me outweigh the inconveniences of the you!

Bear in mind, inconveniencing 4.8% of users, does not map identically.

Instead, you are often dumping 4.8+4.8+4.8 as you add block methods, with some overlap.

Re: Botspam apocalypse

#220

It's annoying for sure. I deal with abuse at a large scale. I'd recommend: - Rate-limit everything, absolutely everything. Set sane limits. - Rate-limit POST requests harder. Preferably dynamically based on geoip. - Rate-limit login and comment POST requests even harder. Ban IPs that exceed the amount. - Require TLS. Drop TLSv1.0 and TLSv1.1. Bots certainly break. - Require SNI. Do not reply without SNI (nginx has 44…

I'm sure these would work but I'll probably got banned too just because I often try to poke ip addresses directly. I also often use VPN especially when outside, so I'll definitely got banned.
Post reply on HN