Live data from Hacker News

Botspam apocalypse

memex.marginalia.nu

151–160 of 358 posts

Re: Botspam apocalypse

#151
post #11

Crypto currency mining could be the solution. If one request to the site generates more revenue than it costs in resources, the bot problem is solved. The author says that he is getting 15 bot requests to his site per second. That is about 36 million requests per month. How much does it cost to serve those? $1000 would seem high. $1000/36M = $0.00003 per request. How long would a crypto currency, that is suitable for…

Satoshi Nakamoto almost certainly isn't Adam Back.

It might be enough for the request to require more resources from the requestor than from the server, even if it doesn't actually give the server any money. I mean the requestor probably isn't going to be willing to dedicate more hardware "horsepower" to taking your search engine down than you are to keeping it up. That was the idea behind Hashcash.

As for coins, the current Bitcoin hashrate is about 200 exahashes per second, down from a high of over 250 a couple of months ago, and the block reward is 6.25 BTC until probably June 02024. At a price of US$24000/BTC that's US$150k per block (plus a much smaller amount in transaction fees) or about US$1.25e-18 per hash. So your suggestion of US$3e-5 would require about 2e13 hashes. https://en.bitcoin.it/wiki/Non-specialized_hardware_comparis... says an overclocked ATI Radeon HD 6990 can do about 800 megahashes per second (8e8) so you're looking at about 3e4 seconds of compute on that card, about 8 hours.

Maybe one of the altcoins that uses a hash function with a smaller ASIC speedup would be a better fit, although I don't know enough about mining to know if there are any where GPUs are still competitive. Still, it seems like it might be more than a few seconds?

Re: Botspam apocalypse

#152
post #11

Crypto currency mining could be the solution. If one request to the site generates more revenue than it costs in resources, the bot problem is solved. The author says that he is getting 15 bot requests to his site per second. That is about 36 million requests per month. How much does it cost to serve those? $1000 would seem high. $1000/36M = $0.00003 per request. How long would a crypto currency, that is suitable for…

As much as I hate the whole cryptocurrency hype myself, I think I agree that a proof-of-work requirement on spam detection that pays in the hosts favour could help solve spam to some degree.

Before bitcoin, there was hashcash, which aimed to do exactly this: http://www.hashcash.org/ . The original bitcoin paper cites it, in fact.

Re: Botspam apocalypse

#154

I am running a website builder with > 20K sites. I use open contact forms without captcha. What worked for me is to use a one line javascript that places current timestamp in a hidden input field that is default 0. Then I check on the backend and if the value is either 0 or time to fill out and send the form is less than 4 seconds I block as spam. This blocks more than 99% of spam and also takes care of most human co…

If you don't want to require users to run javascript you should be able to make the server generate the timestamp.

How do you do that, without bots being able to circumvent the feature?

Re: Botspam apocalypse

#155

Earlier quoted context omitted.

Yup, in adtech, "IP is an AWS block" was a bot 99.999% of the time. The 0.001% was that person using EC2 as a proxy or VPN server.

It's not only AWS. Also happens on Azure and GCP.

True, but at the time, 3 - 4 years ago, Azure and GCP IPs were minimal.

Guess the fraudsters were vendor locked lol.

Re: Botspam apocalypse

#156
post #147

It's annoying for sure. I deal with abuse at a large scale. I'd recommend: - Rate-limit everything, absolutely everything. Set sane limits. - Rate-limit POST requests harder. Preferably dynamically based on geoip. - Rate-limit login and comment POST requests even harder. Ban IPs that exceed the amount. - Require TLS. Drop TLSv1.0 and TLSv1.1. Bots certainly break. - Require SNI. Do not reply without SNI (nginx has 44…

> If you can, require HTTP/2.0. Bots break. Non-bots break as well. I have Firefox configured to use HTTP/1.1 only. No reason to chase Google's standard-of-the-day, HTTP/1.1 has worked for ages and it will continue to do so for the foreseeable future.

Some old browsers break as well, if it's worth it depends on the website. It's your prerogative to disable an useful feature, you can also disable JavaScript. But there's little reason for a website operator to cater to that unnecessary edge case if it's mostly used for abuse.

Re: Botspam apocalypse

#157
post #147

It's annoying for sure. I deal with abuse at a large scale. I'd recommend: - Rate-limit everything, absolutely everything. Set sane limits. - Rate-limit POST requests harder. Preferably dynamically based on geoip. - Rate-limit login and comment POST requests even harder. Ban IPs that exceed the amount. - Require TLS. Drop TLSv1.0 and TLSv1.1. Bots certainly break. - Require SNI. Do not reply without SNI (nginx has 44…

> If you can, require HTTP/2.0. Bots break. Non-bots break as well. I have Firefox configured to use HTTP/1.1 only. No reason to chase Google's standard-of-the-day, HTTP/1.1 has worked for ages and it will continue to do so for the foreseeable future.

That seems like a strange reason to me. Isn't HTTP/2.0 faster? Isn't it also basically transparent to the end user?

I'm trying to figure out what I would gain by configuring my browser to use HTTP/1.1 only.

Re: Botspam apocalypse

#158
post #147

Earlier quoted context omitted.

> If you can, require HTTP/2.0. Bots break. Non-bots break as well. I have Firefox configured to use HTTP/1.1 only. No reason to chase Google's standard-of-the-day, HTTP/1.1 has worked for ages and it will continue to do so for the foreseeable future.

Genuinely curious, why disable HTTP2? Your web browsing must be awfully slow sans multiplexing.

> why disable HTTP2

Because it adds nothing to improve my browsing experience, and reducing the number of protocols supported by my browser from 3 to 1 also reduces the attack surface.

> Your web browsing must be awfully slow sans multiplexing.

And yet it's not slowed down at all. How many different resources must a web page use before it feels slow on a connection pool of keep-alive TCP sockets? Maybe people visit some wild experimental web pages with hundreds of blocking tags that are not bundled/minified?

Either way, my experience is it doesn't slow anything down when I use both websites (forums, resources, youtube, social media) and web apps (banking, maps, food delivery etc).

Re: Botspam apocalypse

#159
post #2

> They're a major part in killing off web forums, and a significant wet blanket on any sort of fun internet creativity or experimentation. > The only ones that can survive the robot apocalypse is large web services. Your reddits, and facebooks, and twitters, and SaaS-comment fields, and discords. They have the economies of scale to develop viable countermeasures, to hire teams of people to work on the problem full ti…

> The solution is real simple

Uhhmmm, I beg to differ and so do a lot of very smart people with many more servers and users than you or I are likely to see.

As with most 'Oh, its' Simple - Just Do XYZ' solutions there are often very good reasons for not doing the 'Easy/Simple/One-Liner' and here are a few with yours -

Firstly - The '10 bux' could exclude a vast swathe of the poorest. Skipping a couple of Starbuck coffees vs. the local currency equivalent of whatever you are charging equating to a month's worth of food or being able to send at least one of your children to the local village school. I mean - your forum / site so you can gate it anyway you wish, I'm just pointing out that it could and would be exclusionary (perhaps unintentionally so).

Next Problem: Accepting and Processing the 'Good Behavior' deposit. Congratulations, you now need to become a Payment Processor and as such have certain legal requirements regarding payment details and storage and also tax returns. 'Oh, just Off-Load it to Stripe' someone might suggest. Do-able I guess but anyone who has taken payments over the internet will tell you that its a Royal Pain in The Ass. Also, now all a 'Griefer' needs to do is run a few dodgy cards through your registration system and 'Poof' there goes your payment processor and/or the fees go sky high.

Most ‘oh its simple – why don’t they just…’ overlook (or are not aware) of the many, many good reasons why greater minds than yours or mine haven’t already implemented it.

Sure – sometimes people do come up with novel solutions to old problems so theres no harm in spit-balling and I’m certainly not directing any scorn or ill-intent in my reply.

Re: Botspam apocalypse

#160
> I can't afford to operate a datacenter to cater to traffic that isn't even human. This spam traffic is all from botnets with IPs all over the world.

In our experience (we don't have a forum), almost all of our bot traffic has been SEO spiders (or claiming to be so).

Post reply on HN