Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Botspam apocalypse

memex.marginalia.nu

Post reply on HN
Print view View original

341–350 of 358 posts

Re: Botspam apocalypse

#341

Post by TrackerFF »

JohnJamesRambo wrote: post #25

What is the reason behind bots spamming marginalia? What’s the motivation? What do they gain? I always wonder about these things.

Not OP, but:

Many of these bots are exceptionally simple - zero tailoring, just scriptkiddie copy/past/run stuff.

They could simply iterate through a list of websites inputted by the user. Said list could be some curated list you find on blackhat forums, which is just a collection of websites with traffic over a certain threshold. Doesn't say anything about the content or what type of website, just the URL.

Then it does a simple operation to map the website - as well as checking for input forms. If an input form is detected, it does a POST operation. If there's an error, timeout, or whatever, it simply moves to the next one.

This of course sucks if you try to run a search engine - because even the simplest of bots will succeed when your website is literally just one website, with the text/input field right there in the front, and with no captcha or similar to dissuade legit users.

Re: Botspam apocalypse

#342

Post by sunsky8 »

tmikaeld wrote: post #328
dj_mc_merlin wrote: post #205

Earlier quoted context omitted.

I think the it depends on what counts as a "request" in terms of pricing. Is it only successful checks? Pricing would be fine then. If it also includes failed checks then there is no point in the service, including the Advanced plan. Would eat through the entire credit in a day.

If it was on successful validations, they would called it so, no it’s on every request, even failed ones.

I had a look into their terms and they seem to count requests but charge for successful validations.

Re: Botspam apocalypse

#343

Post by JZerf »

andai wrote: post #305
Avamander wrote: post #108

It's annoying for sure. I deal with abuse at a large scale. I'd recommend: - Rate-limit everything, absolutely everything. Set sane limits. - Rate-limit POST requests harder. Preferably dynamically based on geoip. - Rate-limit login and comment POST requests even harder. Ban IPs that exceed the amount. - Require TLS. Drop TLSv1.0 and TLSv1.1. Bots certainly break. - Require SNI. Do not reply without SNI (nginx has 44…

>Bots break. Wonder if you could respond in a way to get them to crash, or even better, to hang indefinitely.

If you're reasonably certain that the request is being made by a malicious bot, you could try responding with a ZIP bomb [1] if HTTP 1.1 or newer is being used and the bot accepts compression. It can cause problems for several browser engines.

[1] https://blog.haschek.at/2017/how-to-defend-your-website-with...

Re: Botspam apocalypse

#344

Post by SimplGy »

So you actually mentioned web3, but not the technology from it that is most interesting to me for this problem.

I’m hopeful that “decentralized identity” is a solution here.

In theory this would let people cryptographically prove whatever small fact about themselves they would like to share with a service provider like marginalia, without paying you, or sharing their identity, or anything else.

Eg: “I am a human”, or “I have written less that 100k words on the Internet” or “my comments have an upvote average above zero”

Re: Botspam apocalypse

#345

Post by piaste »

eropple wrote: post #277
bluedino wrote: post #214

Earlier quoted context omitted.

You used to have to really, really try to get banned or even probated on SA, now it doesn't take much at all.

I can't remember the last time I saw a ban on SA that wasn't after a string of "stop trashing discussion" probes--sometimes in the dozens--or wasn't some flavor of FYAD-escapee bigot or death-threat-spewing weirdo. Even extremely tedious, thread-killing arguers will often be left alone unless a IK is ignored when they say "drop it" or whatever, and that's usually just a sixer. There are people who get really mad that…

> I can't remember the last time I saw a ban on SA that wasn't after a string of "stop trashing discussion" probes--sometimes in the dozens--or wasn't some flavor of FYAD-escapee bigot or death-threat-spewing weirdo.

Well, you made me curious so I looked at the leper's colony and looks like things haven't changed much:

- Ban for offensive joke: https://forums.somethingawful.com/showthread.php?goto=post&p...

- Ditto: https://forums.somethingawful.com/showthread.php?goto=post&p...

- Ban for insulting a political movement: https://forums.somethingawful.com/showthread.php?goto=post&p...

That was just on the first page, mind you.

There were some funny bans in there, though. "No, climate change does not justify pedophilia" and "Do not ever ask how to convert commercial drones to deploy grenades" are my favourites.

Re: Botspam apocalypse

#346

Post by dspillett »

b112 wrote: post #219
dspillett wrote: post #216

Earlier quoted context omitted.

If doing something fends off a lot of bots, but also inconveniences a very small number of people who have significantly non-standard or just out-of-date configurations, I'm likely to favour protecting myself from the former over worrying about the latter. To paraphrase Mr Spok: The inconveniences of the me outweigh the inconveniences of the you!

Bear in mind, inconveniencing 4.8% of users, does not map identically. Instead, you are often dumping 4.8+4.8+4.8 as you add block methods, with some overlap.

FWIW I don't block non-HTTP2, but at some point that 4.8% who can't use it will be much smaller and if you are still turning it off you will be part of an increasingly insignificant population.

It is not dissimilar to measures that will have the side effect of stopping apps working on IE - I wouldn't in DayJob¹ but I'd not think twice on a personal project. I don't have time to care that much about people with very non-standard or very out-of-date setups, if accidentally blocking them is a side effect of a benefit I might feel. Added benefit: blocking them effectively removes potentially difficult users from the support queue!

And to some, that 4.8%+4.8%+… doesn't overlap their target audience enough that losing the few in both categories isn't really a problem in the grand scheme of things.

----

[1] some of our clients still have users using IE11 even though we no longer officially support it² so we still don't want anything other than minor display & performance issues there ATM

[2] of course we will for more money, if someone wants to pay us to address said minor display & performance issues

Re: Botspam apocalypse

#347

Post by remram »

tpoacher wrote: post #122
remram wrote: post #40

Earlier quoted context omitted.

> bots hammering the web service with requests and consuming resources I've never seen this referred as "spam". Denial of service, botting, scraping, sure, but does anyone call that spam?

It's spam from a server owner's point of view in the broader sense, in that it is "junk requests" instead of legitimate requests, they can be sent as a flood at no cost or consequence to the senders, and it's up to you as the recipient to find a way to filter it all to separate the wheat from the chaff. It's certainly not denial of service, that means something far more specific. One could call it "scraping", but I'd…

> And "botting" is not a word.

That's fair. Replace with "scripting" if preferred.

Re: Botspam apocalypse

#348

Post by me_again »

birracerveza wrote: post #34
me_again wrote: post #6

Earlier quoted context omitted.

That seems "prohibitively convoluted" to me, if nothing else.

Because you don't have experience with it. There's nothing complicated about SIWE, minting an NFT and checking its validity, certainly not to describe it "prohibitively convoluted" aside from being scared of web3 keywords. Come on now. Not commenting on op's solution's validity or effectiveness, just replying to your comment.

If you rounded up 100 regular web users and ran a usability test where they had to do this to post a comment, how many do you think would succeed?

Re: Botspam apocalypse

#349

Post by lifeisstillgood »

denton-scratch wrote: post #194
lifeisstillgood wrote: post #148

Earlier quoted context omitted.

>>> Automation is not reserved for those providing "services". It also should be available to those using them. Yes ! I call this software literacy. And yes - no matter how cool the JS on a major site, the fact that the sites goals are to keep me there and clicking and my goals are to get what I want with minimal action are in conflict. I would suggest that bots are actually not a problem. For most things I would lik…

> Telling me as and when that I need to visit the dentist Isn't that simply your calendar? Sure, you want it automated; but it doesn't need internet access, it doesn't need to crawl or search, I don't know why you refer to it as a 'bot'. To my mind, the idea of personal 'bots' was that you could give it some general instructions such as "Let me know when the content at any of these URLs changes", and then leave it ru…

Not really. I want a PA, who will help organise my day, monitor my emails and messages, arrange meetings, prepare materials, tell me options for kids camps, you know what people who can afford a full time helper get. But I don't want to spend 80k pa or more. I want a bot.

Re: Botspam apocalypse

#350

Post by david_draco »

GTP wrote: post #78
david_draco wrote: post #64

Have a "CAPTCHA" that gives the IP reputation for some time (cookie+IP=key), but instead of a CAPTCHA make the web page / browser solve and submit a BOINC task from a randomly picked science project. No user interaction needed, it has the benefits of "paying by computation" of cryptocurrencies without the tracing, and if bots solve the problem efficiently, it's good for science.

But solving a BOINC task requires too much time while the average user rightfully expects a webpage to load within 5 seconds or so

Usually you have a landing page, and then you enter stuff there, and finally you get output. That gives you about 1 minute before returning first results.

If the user is faster, you can show something like cloudflare does when you visit through Tor.

On later submissions you can reuse the reputation from the cookie.

Post reply on HN