Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Botspam apocalypse

memex.marginalia.nu

Post reply on HN
Print view View original

61–70 of 358 posts

Re: Botspam apocalypse

#61

Post by TheCapeGreek »

hamilyon2 wrote: post #5

I experienced this firsthand with government immigration websites. The thing is there are only so many time slots and and people are forsed to use a certain web site to apply, so everyone is hunting for available time and generally none are available. So, some creative people set up bots which check periodically for them. They are paid services which will do that for you. Now we have bots hammering gatekeeper's websi…

It gets more fun when there are arbitrary restrictions on process. Fun anecdote: South Africa's Home Affairs website, where you make bookings for passports, only lets you book appointment dates 2 weeks ahead normally. It's effectively permanently booked even without bots this way.

Luckily, if you're technically inclined, editing the value of the input element via dev tools is accepted by the form.

Re: Botspam apocalypse

#62

Post by Nextgrid »

sircastor wrote: post #32
paulmd wrote: post #2

> They're a major part in killing off web forums, and a significant wet blanket on any sort of fun internet creativity or experimentation. > The only ones that can survive the robot apocalypse is large web services. Your reddits, and facebooks, and twitters, and SaaS-comment fields, and discords. They have the economies of scale to develop viable countermeasures, to hire teams of people to work on the problem full ti…

>The solution is real simple: it costs 10 bux to register an account, if you're a nuisance your account is banned and you pay 10bux to get back on. Many years ago there was a public server called SDF (Super Dimensional Fortress). It was a BSD system and anyone could get a user account for $1. The theory was even the least of us, a kid scrounging for money on the street, could come up with a dollar (and presumably the…

> As time went on, the number of people interested in this was dwindling.

That's mostly because access to computers has became easier - you can either get your own Linux box or get a proper VPS for extremely cheap (if not free - see cloud provider free tiers) nowadays so why bother with a non-root account on a BSD system?

IMO it doesn't have anything to do with the barrier to entry.

Re: Botspam apocalypse

#63

Post by BasiliusCarver »

BiteCode_dev wrote: post #31

It's not that bad. First, of course, you have cloudflare and recaptcha, which are free and very efficient, as the author say. But even if you don't want to use them (some of my services don't), most bots are very dumb: - require JS, and you lose half of the web ones - silly tricks like hidden input fields in forms that worked in 2000 still work in 2022. Use a bunch of them, and you can yet again halve the bot traffic…

One of the things I’ve done before among the other suggestions is to put a hidden link like /followmeifyouscraping.html in the landing page to get a bit of info about scraping volume and then you can use fail2ban filters to block if it’s visited if you want

Re: Botspam apocalypse

#64

Post by david_draco »

Have a "CAPTCHA" that gives the IP reputation for some time (cookie+IP=key), but instead of a CAPTCHA make the web page / browser solve and submit a BOINC task from a randomly picked science project. No user interaction needed, it has the benefits of "paying by computation" of cryptocurrencies without the tracing, and if bots solve the problem efficiently, it's good for science.

Re: Botspam apocalypse

#65

Post by paulmd »

sircastor wrote: post #32
paulmd wrote: post #2

> They're a major part in killing off web forums, and a significant wet blanket on any sort of fun internet creativity or experimentation. > The only ones that can survive the robot apocalypse is large web services. Your reddits, and facebooks, and twitters, and SaaS-comment fields, and discords. They have the economies of scale to develop viable countermeasures, to hire teams of people to work on the problem full ti…

>The solution is real simple: it costs 10 bux to register an account, if you're a nuisance your account is banned and you pay 10bux to get back on. Many years ago there was a public server called SDF (Super Dimensional Fortress). It was a BSD system and anyone could get a user account for $1. The theory was even the least of us, a kid scrounging for money on the street, could come up with a dollar (and presumably the…

> Many years ago there was a public server called SDF (Super Dimensional Fortress). It was a BSD system and anyone could get a user account for $1. The theory was even the least of us, a kid scrounging for money on the street, could come up with a dollar (and presumably the postage to mail it). To a certain person, access to this kind of server was invaluable - the only situation you could hope to get close to this kind of system. As time went on, the number of people interested in this was dwindling.

SDF is still around though, and still operates on the same model - pay once to get in, and you can stay as long as you want unless you become a nuisance.

> Jumping through hoops is a useful gateway, but if your hoops are too complex or arduous, you miss out on people who you genuinely want to include in your community.

It is certainly not impossible for communities with this model to die - that's not what I'm saying at all. Small social media sites die all the time, including with Reddit-style gamification bullshit. Or they turn into cesspits like Digg or Voat.

But yes, increasing the friction of engagement is literally the point, you are losing some users but increasing the quality of the ones who remain. It's the old "fire your bad customers" routine, but for social media.

"Oh no, we are all losing out on your valuable shitposting, how will this community ever go on?"

Re: Botspam apocalypse

#66

Post by SyneRyder »

onefuncman wrote: post #28
JohnJamesRambo wrote: post #25

What is the reason behind bots spamming marginalia? What’s the motivation? What do they gain? I always wonder about these things.

I want to run a honeypot for doing more research on bots and the economics for them, but I get bogged down quickly in the planning stages. I should just start with a vulnerable wordpress site or something.

Just make a site with a Contact page, with a comment form that logs the details of every request (IP address, timestamp, message content, email provided). You'll get plenty of data for research, once the page has been indexed into the database the comment form spammers use. For bonus points, put the contact form at the bottom of every page of your website.

A couple of my toy/project websites accidentally became honeypots. Rather than shut down the comment forms, I now have those sites generate summary logfiles that I can upload daily to AbuseIPDB.

EDIT: Forgot to mention, also log the Referer field and User-Agent on each request. Very, very useful information for research and detection.

Re: Botspam apocalypse

#67

Post by realaravinth »

tmikaeld wrote: post #23
timmaxw wrote: post #21

I wonder if proof-of-work would help. Suppose every form submission requires an expensive calculation, calibrated to take about 1 second on a typical modern computer/smartphone. For human users, this happens in the background, although it makes the website feel slower. But for bots, it dramatically limits how many submissions each botnet host can make to random websites.

"mCaptcha uses SHA256 based proof-of-work(PoW) to rate limit users." https://github.com/mCaptcha/mCaptcha

Hello!

I'm the author of mCaptcha, I'd be happy to answer any questions that people might have :)

Re: Botspam apocalypse

#68

Post by philprx »

Joel_Mckay wrote: post #36

For small sites, I would just use a simple firewall: 1. whitelist the finite IP ranges for the regional ISPs/country where you do business 2. blacklist the proxy and tor exit nodes 3. blacklist the list of published compromised servers 4. add spamhaus blacklists 5. add fail2ban rules to trip on common server security scans, and unused common service ports 6. publicly reply to those having access issues, and imply the…

Tor users are often legitimate good internet citizens.

A lot of (lucky) us have the luxury to live in real democracies.

Some others live in countries that use every single aspect of their private lives (DPI, mass surveillance) to put pressure on them and bend them to the regime's will.

In my opinion, Tor and anonymity should not be killed as a result of silly bots.

Re: Botspam apocalypse

#69

Post by GTP »

paulmd wrote: post #60
paulmd wrote: post #2

> They're a major part in killing off web forums, and a significant wet blanket on any sort of fun internet creativity or experimentation. > The only ones that can survive the robot apocalypse is large web services. Your reddits, and facebooks, and twitters, and SaaS-comment fields, and discords. They have the economies of scale to develop viable countermeasures, to hire teams of people to work on the problem full ti…

few edits I wanted to make but couldn't while HN was down: this comes to a question of intentions, right? Like are you trying to build a high-value community , or are you trying to make a billion-dollar company? Photrio or Pentaxforums is never going to sell for a billion dollars like Reddit, and that's not the kind of community that Reddit is trying to build. The highly-chaotic multithreaded model of Reddit/HN/etc i…

You're right that requiring a small fee to register can solve the problem for small communities (and so OP is wrong in the paragraphs that you quoted), but still this solution doesn't work for OP's case as this requires login and he says that he doesn't want to know who is using his search engine (I guess for privacy reasons) and in this case indeed the bots can harm the proliferation of this kind of small scale services.

Re: Botspam apocalypse

#70

Post by rapnie »

david_draco wrote: post #64

Have a "CAPTCHA" that gives the IP reputation for some time (cookie+IP=key), but instead of a CAPTCHA make the web page / browser solve and submit a BOINC task from a randomly picked science project. No user interaction needed, it has the benefits of "paying by computation" of cryptocurrencies without the tracing, and if bots solve the problem efficiently, it's good for science.

That is a nice idea. So bit similar to mCaptcha [0] that uses PoW algorithm, mentioned in other comment [1] in the thread.

[0] https://mcaptcha.org/

[1] https://news.ycombinator.com/item?id=32339902

Post reply on HN