Live data from Hacker News

Botspam apocalypse

memex.marginalia.nu

281–290 of 358 posts

Re: Botspam apocalypse

#281

For my forum with 500k users a month I just added a registration captcha related to my niche. E.g. for a Dark Souls forum it would say "what game is this forum about?" And if you got it wrong the validation would include "tip it's just two words D rk S*ls". This reduced spam by over 99% and didn't annoy people with recaptcha. If someone was unable to get past that captcha (it still happens I have logs!) I figured the…

Reminds me of a guy who implemented a pre-screen on his phone calls to stop spammers. He said he wanted to use something simple at first and that he planned to tweak it depending upon how many spammers go through. So his phase one it asks "Dial 1 to continue". But that was enough to stop all the spam calls so he never had to improve it.

Re: Botspam apocalypse

#282
tangential : Two weeks ago (and for a while) our country's twitter-sphere (Chile) was completely and obviously dominated by bots, they were starting and inflating trending topics with absurd lies, spreading fear and chaos in favor of "Rechazo" (the option against our new constitution in the next ballot) or echo chambers for republican and extreme right associated politicians.. What happened? a self organized group[1] started to do data analysis of the trending topics and delivering the results to the people showing who was behind the campaigns and synthetic likes, after this, prominent and public figures from that sector started to cut funding for bot networks (because of the public shaming and media attention they were receiving) and is so pathetic now ,they can't even get more than 100 likes and often the most popular response is a refutation or the very same analysis showing the bot network working with substantially more organic likes. I think is a very interesting phenomenon to watch. Note that this lies/fear/chaos campaign is transversal, from rural AM radio to tiktok, but is not working at all. People is very aware of these campaigns and knows how to defend against. Truth is stronger than money.

- [1] https://twitter.com/BotCheckerCL

Re: Botspam apocalypse

#283

Earlier quoted context omitted.

"This spam traffic is all from botnets with IPs all over the world. Tens, maybe hundreds of thousands of IPs, each with a relatively modest query rates, so rate limiting does all of bupkis."

Yep, there isn't a silver bullet that curtails all abuse.

Blocking the entire aws/gcp/azure/digital ocean/linode IP ranges will stop 99.999% of malicious bot traffic full stop.

Re: Botspam apocalypse

#284

I am running a website builder with > 20K sites. I use open contact forms without captcha. What worked for me is to use a one line javascript that places current timestamp in a hidden input field that is default 0. Then I check on the backend and if the value is either 0 or time to fill out and send the form is less than 4 seconds I block as spam. This blocks more than 99% of spam and also takes care of most human co…

I also use essentially the same technique (although I have the server generate the timestamp instead of using JavaScript) on my website and concur that this is a highly effective technique for blocking bot submissions.

Re: Botspam apocalypse

#285
post #140

I am running a website builder with > 20K sites. I use open contact forms without captcha. What worked for me is to use a one line javascript that places current timestamp in a hidden input field that is default 0. Then I check on the backend and if the value is either 0 or time to fill out and send the form is less than 4 seconds I block as spam. This blocks more than 99% of spam and also takes care of most human co…

I like this solution because spammers are unlikely to try to get around it. A delay eats into their time budget and they can't introduce a human-like waiting time on every site they try to spam, better to just move on to find cheaper targets.

You could just decrease the timestamp instead of actually waiting.

Re: Botspam apocalypse

#286

It's annoying for sure. I deal with abuse at a large scale. I'd recommend: - Rate-limit everything, absolutely everything. Set sane limits. - Rate-limit POST requests harder. Preferably dynamically based on geoip. - Rate-limit login and comment POST requests even harder. Ban IPs that exceed the amount. - Require TLS. Drop TLSv1.0 and TLSv1.1. Bots certainly break. - Require SNI. Do not reply without SNI (nginx has 44…

> Require TLS. Drop TLSv1.0 and TLSv1.1. Bots certainly break. So will people who run older computers with older software. But I guess people who don't have money don't matter for commercial websites so screw 'em.

I don't think there's much web you can visit with those browsers anyway. Windows XP with IE and no SNI support, maybe sites from that era without JavaScript would work?

Re: Botspam apocalypse

#287
I will reiterate what I had been saying on HN for years:

1) The problem is centralization. Yes DNS is federated but there is a central registry. This means anyone can spam you@yourdomain.com or visit your web server listening for HTTP connections at www.domain.com

2) DNS is a glorified search engine. Human readable domain names are only needed for dictating a domain name (and listeners often make mistakes anyway). They only map to a small fraction of URLs) namely the ones with “/“ path name. For most others, the human readability adds little benefit.

3) Start using URIs that are not human readable. The titles, favicons and other metadata of resources should simply be cached, and displayed to the user in their own bookmarks, search engines or whatever. For Javascript environments, variables can easily hold non human readable URIs. Also QR codes can resolve to non human readable URIs.

4) There may be some cookie policy for third party hostnames etc. but just make them non human readable also.

5) We should have DHT or other decentralized systems for routing, and here is the key… in this system, you need a capability issued by the website / mailbox owner in order for your message to be routed to them. If the capability is compromised and used to get a ton of SPAM, they simply revoke that specific capability (key).

For HTTP websites you can already implement it on your side by signing the keys / capabilities ie session cookie balues with an HMAC, and there is no need to even do network I/O to verify them, you can upload the whitelist to the edges and check them there easily.

But going further, for new routing protocols, IP addresses should be removed after the first hop in the DHT, because the global routing system will send traffic there otherwise. See how SAFE network does it.

6) I don’t need a “real names policy” or “blue checkmark”. I can know who “The Real Bill Gates (TM)” is through some verified claims by Twitter or someone else. Just because I have the email billgates@microsoft.com doesnt mean I should be able to email him. There can be many Bill Gates. The names are just verified claims by some third party. Here on HN we dont have names or photos, and it works just fine.

7) Most of the celebrity culture, papparazzi, Elon Musk and Donald Trump moving markets and tweeting at 5am to 5 million people at once, are problems of centralization. Both a 1 to many megaphone and a many to 1 inbox. Citizens United is just a symptom of the problem. I have spoken about this (privately owning access to an audience) with Noam Chomsky in an interview I did a year ago:

https://community.qbix.com/t/freedom-of-speech-and-capitalis...

Fox News (Rupert Murdoch), CNN (Ted Turner), Twitter (Elon or Jack), Facebook (Zuck) are controlled by only a few people. Channels on youtube, telegram, podcasts etc are controlled by a few people. This leads to divisions in society, as outrage clickbait rises to the top. Nonprofit models based on collaboration like Wikipedia, Wikinews, Open Source and Science produce far more balanced and benign information for the public.

In short we need alternatives to celebrity culture, DNS and other systems that centralize decision making in the hands of a few, or create firehoses and megaphones. Neither the celebrity nor the public actually enjoy the results.

Re: Botspam apocalypse

#288
post #118

For my forum with 500k users a month I just added a registration captcha related to my niche. E.g. for a Dark Souls forum it would say "what game is this forum about?" And if you got it wrong the validation would include "tip it's just two words D rk S*ls". This reduced spam by over 99% and didn't annoy people with recaptcha. If someone was unable to get past that captcha (it still happens I have logs!) I figured the…

A niche dark souls forum sounds interesting, any chance I could get a link?

You misread the post. That was just an example. A niche dark souls forum wouldn't have 500k users lol.

Re: Botspam apocalypse

#289
post #2

> They're a major part in killing off web forums, and a significant wet blanket on any sort of fun internet creativity or experimentation. > The only ones that can survive the robot apocalypse is large web services. Your reddits, and facebooks, and twitters, and SaaS-comment fields, and discords. They have the economies of scale to develop viable countermeasures, to hire teams of people to work on the problem full ti…

> The solution is real simple Uhhmmm, I beg to differ and so do a lot of very smart people with many more servers and users than you or I are likely to see. As with most 'Oh, its' Simple - Just Do XYZ' solutions there are often very good reasons for not doing the 'Easy/Simple/One-Liner' and here are a few with yours - Firstly - The '10 bux' could exclude a vast swathe of the poorest. Skipping a couple of Starbuck cof…

> Firstly - The '10 bux' could exclude a vast swathe of the poorest. Skipping a couple of Starbuck coffees vs. the local currency equivalent of whatever you are charging equating to a month's worth of food or being able to send at least one of your children to the local village school. I mean - your forum / site so you can gate it anyway you wish, I'm just pointing out that it could and would be exclusionary (perhaps unintentionally so).

It is intentionally exclusionary. Not necessarily of the poorest among us, but of those who expect free service. Botters and spammers are disproportionately likely to look for free service. Pretty much any level of required spending in any currency will have a similar effect. By cutting off the abuse-prone free tier that many bad actors depend on, you dramatically decrease your exposure to abuse.

The point is not to keep out the poor people. The point is to make it far more work to get over the hurdle than it's worth for abusers. If you have a way to do the latter without the former that doesn't hinge on pushing a bunch of extra work onto administrators, I suspect quite a lot of people would be very curious to hear about it.

Re: Botspam apocalypse

#290

It's annoying for sure. I deal with abuse at a large scale. I'd recommend: - Rate-limit everything, absolutely everything. Set sane limits. - Rate-limit POST requests harder. Preferably dynamically based on geoip. - Rate-limit login and comment POST requests even harder. Ban IPs that exceed the amount. - Require TLS. Drop TLSv1.0 and TLSv1.1. Bots certainly break. - Require SNI. Do not reply without SNI (nginx has 44…

I'm not very familiar with all the workings of HTTP/2.0 - why would it break bots? Assuming no CloudFlare type protection, does it somehow stop someone from using curl to get (non-JS generated) content? Does it thwart someone accessing the site from something like playwright/selenium?
Post reply on HN