Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Botspam apocalypse

memex.marginalia.nu

Post reply on HN
Print view View original

301–310 of 358 posts

Re: Botspam apocalypse

#301

Post by [deleted] »

ajsnigrutin wrote: post #173
Avamander wrote: post #108

It's annoying for sure. I deal with abuse at a large scale. I'd recommend: - Rate-limit everything, absolutely everything. Set sane limits. - Rate-limit POST requests harder. Preferably dynamically based on geoip. - Rate-limit login and comment POST requests even harder. Ban IPs that exceed the amount. - Require TLS. Drop TLSv1.0 and TLSv1.1. Bots certainly break. - Require SNI. Do not reply without SNI (nginx has 44…

> - Rate-limit everything, absolutely everything. Set sane limits. This breaks when multiple users are behind the same IP. I've seen services fail even in classroom, because the prof did something and a few tens of students followed (captchas everywhere).

[deleted]

Re: Botspam apocalypse

#302

Post by rndgermandude »

annoyingnoob wrote: post #280
CodeSgt wrote: post #222

Earlier quoted context omitted.

> Rate-limit login and comment POST requests even harder. Ban IPs that exceed the amount Don't ban IPs. Or if you do, let the ban expire relatively quickly (days/weeks, not months/years).

I ban IPs from small data centers all the time. For my purposes there is no need to support traffic from small hosting providers that are everywhere all over the world. I do not tend to ban the IPs of commercial ISPs that provide service to end users.

You will probably ban a lot of VPN users as collateral damage. VPN providers often use these small and relatively cheap providers for their endpoints.

You may be fine with banning those VPN users, or even want that - lots of bots will try to hide behind "legitimate" VPNs - but one has to be aware of this consequence at least, especially considering that more and more people seem to use them - probably also thanks to the aggressive "sponsoring" certain providers such as ExpressVPN do on e.g. a wide variety youtube videos.

Re: Botspam apocalypse

#303

Post by megous »

RL_Quine wrote: post #223
viraptor wrote: post #94

> If Marginalia Search didn't use Cloudflare, it couldn't serve traffic. Cloudflare is not the only CDN/protection. It's the most popular and the most evil one. You have a choice.

Why do you consider them to be "the most evil"? Their services seem to be completely fine in almost every regard, and their communication doesn't at all suggest that they might be evil.

Not for the website visitors.

Re: Botspam apocalypse

#304

Post by JZerf »

pphysch wrote: post #274

The only real solution to the abuse of anonymous protocols is to stop using anonymous protocols and use protocols where clients can be held accountable. But that's politically nonviable in the West.

I don't see any good reason why people can't be allowed to remain anonymous while still allowing website operators from taking measures to stop bot abuse. CAPTCHAs can already stop many bots. Other commenters have also mentioned that things like Proof of Work systems and micro-transactions could also stop bot abuse. These don't necessarily require giving up anonymity.

Re: Botspam apocalypse

#305

Post by andai »

Avamander wrote: post #108

It's annoying for sure. I deal with abuse at a large scale. I'd recommend: - Rate-limit everything, absolutely everything. Set sane limits. - Rate-limit POST requests harder. Preferably dynamically based on geoip. - Rate-limit login and comment POST requests even harder. Ban IPs that exceed the amount. - Require TLS. Drop TLSv1.0 and TLSv1.1. Bots certainly break. - Require SNI. Do not reply without SNI (nginx has 44…

>Bots break.

Wonder if you could respond in a way to get them to crash, or even better, to hang indefinitely.

Re: Botspam apocalypse

#306

Post by mjevans »

nottorp wrote: post #139
mjevans wrote: post #52

Earlier quoted context omitted.

I'm one of those radical militants who refuses to give up any means of direct contact... However for a small scale thing I'd gladly go visit at a face to face meetup to fulfill this type of validation.

> However for a small scale thing I'd gladly go visit at a face to face meetup to fulfill this type of validation. Even if it were 3 flights totalling 18 hours away? :) Or even just from one coast of the US to another...

Someone that far away shouldn't want my direct contact information to join a group.

However there is a medium / large organization case, where each area has local 'chapters' or some other term for a small fragment of the larger group. In that case the local leaders each operate as a small group for their areas.

Re: Botspam apocalypse

#307

Post by RamRodification »

megous wrote: post #300
RamRodification wrote: post #233

Earlier quoted context omitted.

Off-topic, but isn't that a normal (non-reverse) leaky bucket? When the bucket gets full the rate limiting engages. An empty bucket allows for a burst without getting full. It slowly leaks over time at a rate that allows a normal amount of traffic without filling up.

To me, it's a bucket that's being filled at constant rate from a tap until it's full, and the traffic requires taking some water from the bucket. If there's no water, the trafic has to be dropped or wait in a queue. Basicaly, you can look at it either way.

I like it. I guess it's one of those things where it depends on the example used when you learned about it? For me it was some Nginx guide on rate limiting and I think they described it in the way I see it.

Re: Botspam apocalypse

#308

Post by stonemetal12 »

sirshmooey wrote: post #150
gkbrk wrote: post #147

Earlier quoted context omitted.

> If you can, require HTTP/2.0. Bots break. Non-bots break as well. I have Firefox configured to use HTTP/1.1 only. No reason to chase Google's standard-of-the-day, HTTP/1.1 has worked for ages and it will continue to do so for the foreseeable future.

Genuinely curious, why disable HTTP2? Your web browsing must be awfully slow sans multiplexing.

https://github.com/dalf/pyhttp-benchmark/blob/master/results...

HTTP2 is barely any better than http 1, if you want it to make a 1/10 of a second difference you have to be making 100s of requests.

Re: Botspam apocalypse

#309

Post by ineptech »

tablespoon wrote: post #247
paulmd wrote: post #2

> They're a major part in killing off web forums, and a significant wet blanket on any sort of fun internet creativity or experimentation. > The only ones that can survive the robot apocalypse is large web services. Your reddits, and facebooks, and twitters, and SaaS-comment fields, and discords. They have the economies of scale to develop viable countermeasures, to hire teams of people to work on the problem full ti…

> This is not true at all. There are web forums that are not "web-scale" and don't spend all day fighting bot spam. The solution is real simple: it costs 10 bux to register an account, if you're a nuisance your account is banned and you pay 10bux to get back on. That doesn't work at all unless your service is already pretty popular. Who would pay $5 to access a new, empty forum? You mention "you need to bootstrap a c…

Step 1: Plant a tree twenty years ago...

Re: Botspam apocalypse

#310

Post by paulmd »

jart wrote: post #45
paulmd wrote: post #2

> They're a major part in killing off web forums, and a significant wet blanket on any sort of fun internet creativity or experimentation. > The only ones that can survive the robot apocalypse is large web services. Your reddits, and facebooks, and twitters, and SaaS-comment fields, and discords. They have the economies of scale to develop viable countermeasures, to hire teams of people to work on the problem full ti…

> it costs 10 bux to register an account, if you're a nuisance your account is banned and you pay 10bux to get back on. You've highlighted its biggest tradeoff which is that it creates an economic incentive to ban people. The only way to make more money, is to have more rules and culture for ostracizing people. It would have been smarter of Something Awful (since that's the site we're talking about) to charge $4/mont…

> It would have been smarter of Something Awful (since that's the site we're talking about) to charge $4/month or something.

The innovative thing off that model is driving the revenue off the misbehavers instead of good citizens. You don't want to have the shitters around even if they are paying $4/month, and you don't want to drive off good-faith users even if they're mediocre/hapless/etc. So run the site off the backs of the people you don't want to have around.

People don't like paying monthly (this is even true of, say, app store revenue today) and if you apply recurring charges then when people don't think they're getting enough value they'll leave. You have a hard enough time on the user-acquisition side, why make it worse on the retention side by driving away the users who you're actually trying to keep?

Billing good citizens works in some situations where you have some specific value that you provide to them - providing sales listings on classifieds boards inside interest-specific forums is a good example, since you are providing access to interested buyers, which is a value-add, same as ebay taking their fee - but just in terms of operating a forum, you aren't a big enough value-add that people are going to pay Netflix-level subscriptions to the Parrot-Ass Discussion Club. You need the users more than they need you at that point. But a one-time fee is viewed much differently by people. People will pay $5 for an app, they aren't going to pay you $5 a month for it though, or at least far fewer.

Post reply on HN