Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Botspam apocalypse

memex.marginalia.nu

Post reply on HN
Print view View original

171–180 of 358 posts

Re: Botspam apocalypse

#171

Post by tlholaday »

gostsamo wrote: post #86
paulmd wrote: post #2

> They're a major part in killing off web forums, and a significant wet blanket on any sort of fun internet creativity or experimentation. > The only ones that can survive the robot apocalypse is large web services. Your reddits, and facebooks, and twitters, and SaaS-comment fields, and discords. They have the economies of scale to develop viable countermeasures, to hire teams of people to work on the problem full ti…

A bit below the author talks how creating any form of account is going counter to their goal of free and anonymous, so they are looking only for a behavioral sift of bots.

Anonymous proof of stake?

Re: Botspam apocalypse

#172

Post by robalni »

mariusor wrote: post #154
robalni wrote: post #146

Earlier quoted context omitted.

If you don't want to require users to run javascript you should be able to make the server generate the timestamp.

How do you do that, without bots being able to circumvent the feature?

People will be able to write programs that circumvent the feature but that's also true for the javascript solution. The point of it was that it gets rid of most spam because most bots fill in the form faster than 4 seconds and are not made to circumvent this feature.

Re: Botspam apocalypse

#173

Post by ajsnigrutin »

Avamander wrote: post #108

It's annoying for sure. I deal with abuse at a large scale. I'd recommend: - Rate-limit everything, absolutely everything. Set sane limits. - Rate-limit POST requests harder. Preferably dynamically based on geoip. - Rate-limit login and comment POST requests even harder. Ban IPs that exceed the amount. - Require TLS. Drop TLSv1.0 and TLSv1.1. Bots certainly break. - Require SNI. Do not reply without SNI (nginx has 44…

> - Rate-limit everything, absolutely everything. Set sane limits.

This breaks when multiple users are behind the same IP. I've seen services fail even in classroom, because the prof did something and a few tens of students followed (captchas everywhere).

Re: Botspam apocalypse

#174

Post by Lex-2008 »

12907835202 wrote: post #109

For my forum with 500k users a month I just added a registration captcha related to my niche. E.g. for a Dark Souls forum it would say "what game is this forum about?" And if you got it wrong the validation would include "tip it's just two words D rk S*ls". This reduced spam by over 99% and didn't annoy people with recaptcha. If someone was unable to get past that captcha (it still happens I have logs!) I figured the…

re: someone was unable to get past that captcha - this reminded me a story I heard back in ICQ times about some human who couldn't pass anti-bot question: "What planet do we live on?"

Re: Botspam apocalypse

#175

Post by Aachen »

mariusor wrote: post #154
robalni wrote: post #146

Earlier quoted context omitted.

If you don't want to require users to run javascript you should be able to make the server generate the timestamp.

How do you do that, without bots being able to circumvent the feature?

Bots can also circumvent this JS thing, so it's the same either way.

    ';
On submit:
Revealing the error condition (that it was submitted too fast) is nice for users and bots alike, of course. Up to you.

I've had websites where I was too fast in submitting a form before. Not any kind of anti-spam, just their server was so fricking slow that I had input the date (iirc it was a reservation system) and clicked next before the JS blobs had finished triggering each other and fully loaded. It would break the page somehow with no visual indication. I found out by looking in the dev console and noticing stuff was still loading in the background. How normal people are able to use the Internet with how often I need the dev console to do entirely ordinary things is a mystery to me.

Re: Botspam apocalypse

#176

Post by TekMol »

kragen wrote: post #151
TekMol wrote: post #11

Crypto currency mining could be the solution. If one request to the site generates more revenue than it costs in resources, the bot problem is solved. The author says that he is getting 15 bot requests to his site per second. That is about 36 million requests per month. How much does it cost to serve those? $1000 would seem high. $1000/36M = $0.00003 per request. How long would a crypto currency, that is suitable for…

Satoshi Nakamoto almost certainly isn't Adam Back. It might be enough for the request to require more resources from the requestor than from the server, even if it doesn't actually give the server any money. I mean the requestor probably isn't going to be willing to dedicate more hardware "horsepower" to taking your search engine down than you are to keeping it up. That was the idea behind Hashcash. As for coins, the…

I have not seen any arguments yet, why Satoshi is not Adam.

I did say "crypto currency, that is suitable for mining in the browser" for exactly this reason: That Bitcoin is not well suited for it.

One would have to look at what typical consumer hardware is good at. Maybe an algorithm that saturates one CPU core with serial calculations that need fast access to exactly 1GB of RAM. I think consumer hardware is pretty good when it comes to single core performance and RAM access.

Re: Botspam apocalypse

#177

Post by Aachen »

> There has been upwards of 15 queries per second from bots. There is just no way to deal with that sort of traffic, barely even to reject it.

If the queries are not a megabit each, you're doing way too much processing before applying rate limiting. Rejecting traffic ought not to take more than 1-2 milliseconds, even if you need to look up an api key or IP address in the database.

I, too, host services on a residential connection: 50 mbps shared with other users. My domains must host hundreds of separate scripts, a few of which have a database attached (I can think of six off the top of my head, but there's over a hundred databases in mariadb so I'm sure there's more that I've forgotten about). This is a ten-year-old laptop with a regular "apt install mariadb", no special configs.

Yes, most traffic is bots, and yes sometimes they submit more than 1 q/s. But it comes nowhere near to exhausting resources to a noticeable extent. Load average is about 0.15, main peaks come from my own cronjobs. If you're having this much trouble rejecting traffic, you might want to spend some time looking at bottlenecks. You'll also notice the bots knock it off if it's unsuccessful.

Re: Botspam apocalypse

#178

Post by Avamander »

ajsnigrutin wrote: post #173
Avamander wrote: post #108

It's annoying for sure. I deal with abuse at a large scale. I'd recommend: - Rate-limit everything, absolutely everything. Set sane limits. - Rate-limit POST requests harder. Preferably dynamically based on geoip. - Rate-limit login and comment POST requests even harder. Ban IPs that exceed the amount. - Require TLS. Drop TLSv1.0 and TLSv1.1. Bots certainly break. - Require SNI. Do not reply without SNI (nginx has 44…

> - Rate-limit everything, absolutely everything. Set sane limits. This breaks when multiple users are behind the same IP. I've seen services fail even in classroom, because the prof did something and a few tens of students followed (captchas everywhere).

A "sane limit" wouldn't be "one person one IP", such a global limit should rather stop one IP (even if it's a nasty CGNAT) from having a negative impact on the entire service.

If such a limit would hinder classroom usage, but that's your target audience then other solutions should be found, fairly logical.

Re: Botspam apocalypse

#179

Post by tmikaeld »

protoduction wrote: post #163
timmaxw wrote: post #21

I wonder if proof-of-work would help. Suppose every form submission requires an expensive calculation, calibrated to take about 1 second on a typical modern computer/smartphone. For human users, this happens in the background, although it makes the website feel slower. But for bots, it dramatically limits how many submissions each botnet host can make to random websites.

I’m the co-founder of Friendly Captcha [0], we offer a proof of work-based captcha since two years or so. Happy to answer any questions. A big part of what makes our captcha successful in fighting abuse is that we scale the difficulty of the proof-of-work puzzle based on the user’s previous behavior and other signals (e.g. minus points if their IP address is a known datacenter IP). The nice thing about a scaling PoW…

While your service is of high quality, the pricing is completely unreasonable for private use cases, many times higher than hosting the site in the first place.

Re: Botspam apocalypse

#180

Post by api »

echelon wrote: post #3

I thought this article was referring to the upcoming deluge of GPT-3/DALL-E bots that will eventually flood all of online discourse. And whatever future models that will be even more indistinguishable from people - perhaps even ones that are good at "signup flow". That's going to be way worse for humanity than spiders and automated scripts sending too much traffic. This article isn't imagining apocalypse creatively e…

We're coming up on the end of open forums and open social media. Everything will require intrusive verification. Anonymous forums could exist but they'll require something else like an anonymous payment, a ton of proof of work on your local machine, etc. to filter out crap.
Post reply on HN