Live data from Hacker News

Botspam apocalypse

memex.marginalia.nu

1–10 of 358 posts

Re: Botspam apocalypse

#2
> They're a major part in killing off web forums, and a significant wet blanket on any sort of fun internet creativity or experimentation.

> The only ones that can survive the robot apocalypse is large web services. Your reddits, and facebooks, and twitters, and SaaS-comment fields, and discords. They have the economies of scale to develop viable countermeasures, to hire teams of people to work on the problem full time and maybe at least keep up with the ever evolving bots.

This is not true at all. There are web forums that are not "web-scale" and don't spend all day fighting bot spam. The solution is real simple: it costs 10 bux to register an account, if you're a nuisance your account is banned and you pay 10bux to get back on.

Even the sites that don't require payment for explicit registration - often succeed by gating functionality or content behind paywalls. Requiring a "premium membership" to post in the classifieds forum is an extremely extremely common thing on small interest-based web-boards (photrio, pentaxforums, homebrewtalk, etc). That income supports the site and supports the anti-bot efforts as a whole. The customer isn't advertisers - it's the community itself, and you're providing the service of high-quality content and access to people with similar interests.

You need to bootstrap a community first, of course, but it doesn't need to be a large community, just a high-value one.

The twitters and facebooks of the world just don't like that solution because they value growth above all other considerations. They'd rather be kings of a billion user website with 200 million bots than a 1k-100k user forum with 100% organic membership and content. And they value engagement over content quality, which is the entire reason comment-tree/vote-based systems have been pushed heavily over web-1.0 threaded forum discussions as well.

This botpocalypse is the inevitable outcome of the systems that social-media giants have created, not inherent outcomes of the internet as a whole.

Re: Botspam apocalypse

#3
I thought this article was referring to the upcoming deluge of GPT-3/DALL-E bots that will eventually flood all of online discourse. And whatever future models that will be even more indistinguishable from people - perhaps even ones that are good at "signup flow".

That's going to be way worse for humanity than spiders and automated scripts sending too much traffic. This article isn't imagining apocalypse creatively enough.

Re: Botspam apocalypse

#4
> The other alternatives all suck to the extent of my knowledge, they're either prohibitively convoluted, or web3 cryptocurrency micro-transaction nonsense that while sure it would work, also monetizes every single interaction in a way that is more dystopian than the actual skull-crushing robot apocalypse.

In the interest of practicality: There's a way to go the web3 route without being laden with transactions:

- Mint a fixed-cost non-transferrable NFT to an address, with ownership limit of 1 per address. - Use SIWE (sign-in with Ethereum) to verify ownership of address & therefore NFT. - If malicious behaviour is detected, mark the NFT as belonging to a malicious actor at the server's end & block the account. - Require non-malicious-marked NFTs in order to use the site/app.

At most, the user only had to perform 1 transaction (minting the non-transferrable NFT) on any blockchain network where the contract resides, & the costs to do so can be made cheaply with Layer 2 networks. (Polygon PoS, Arbtirum, Optimism, zkSync 2.0, etc)

Can this be done entirely without web3? Yes, but the added friction imposed onto malicious actors to generate new addresses & mint new non-transferrable NFTs increases the costs for them considerably.

> If anyone could go ahead and find a solution to this mess, that would be great, because it's absolutely suffocating the internet, and it's painful to think about all the wonderful little projects that get cancelled or abandoned when faced with the reality of having to deal with such an egregiously hostile digital ecosystem.

In all honesty, there's no perfect solution, just hard-to-make tradeoffs: The prevention of botspam inherently requires tracking in some form to resolve said issue, as there's no immediately-recognizable stateless solution for botspam tracking. Someone has to do the tracking to prevent botspam, which inherently involves in state being changed in order to mark an actor as malicious.

Re: Botspam apocalypse

#5
I experienced this firsthand with government immigration websites. The thing is there are only so many time slots and and people are forsed to use a certain web site to apply, so everyone is hunting for available time and generally none are available.

So, some creative people set up bots which check periodically for them. They are paid services which will do that for you. Now we have bots hammering gatekeeper's website. Perhaps hundreds of bots.

Which results in the website is being unavailable, serving a serious qps to bots. I think it is only a matter of time before someone will write bots that will apply to application bots hoping that more entries with their information will provide them with better probability of success.

This is so dystopian and cruel to the average person, and I don't think there is a good solution besides a deep anti-bot expertise whithin the primary website development team

Re: Botspam apocalypse

#6

> The other alternatives all suck to the extent of my knowledge, they're either prohibitively convoluted, or web3 cryptocurrency micro-transaction nonsense that while sure it would work, also monetizes every single interaction in a way that is more dystopian than the actual skull-crushing robot apocalypse. In the interest of practicality: There's a way to go the web3 route without being laden with transactions: - Min…

That seems "prohibitively convoluted" to me, if nothing else.

Re: Botspam apocalypse

#7

I experienced this firsthand with government immigration websites. The thing is there are only so many time slots and and people are forsed to use a certain web site to apply, so everyone is hunting for available time and generally none are available. So, some creative people set up bots which check periodically for them. They are paid services which will do that for you. Now we have bots hammering gatekeeper's websi…

I faced a website like this recently when booking a slot at my own German embassy, went a different route around the embassy instead. What I don't like about the slot system: You won't get a convenient time slot anyways so why do they bother setting it up like this in the first place? Why not just register with your contact details and receive an email with a guaranteed spot at a selection of three days instead. No more need to reload and no need for bots. The Upper Austrian government did this for covid vaccinations in the early phase and it worked very well that way (early, high vaccination rate amongst the elderly and certain professions).

Re: Botspam apocalypse

#8
This kind of botspam is usually pretty easy to address with redbean using the finger https://redbean.dev/#finger and maxmind https://redbean.dev/#maxmind modules. The approach I usually recommend people isn't so much ip reputation, which can be unfair, but rather it allows you to find evidence of clients lying to you. For example, if the User-Agent says it's Windows with a language preference of English, but the TCP SYN packet says it's Linux and MaxMind says it's coming from China, then that means the client is lying (or being MiTM'd) and you can righteously hellban once it's fingered for the crime.

Re: Botspam apocalypse

#9
> has been upwards of 15 queries per second from bots

What type of queries are they generating? For what purpose are querying Marginalia? Scraping and filling internal search engines?

> If anyone could go ahead and find a solution to this mess

I would maybe trying to investigate why are querying your search engine. Is for the search results? Maybe from there you can create and sell an API service. Is for the wiki? Is for research purpose?

I would love to see some data, raw or with some behavior derived from it.

Re: Botspam apocalypse

#10

> The other alternatives all suck to the extent of my knowledge, they're either prohibitively convoluted, or web3 cryptocurrency micro-transaction nonsense that while sure it would work, also monetizes every single interaction in a way that is more dystopian than the actual skull-crushing robot apocalypse. In the interest of practicality: There's a way to go the web3 route without being laden with transactions: - Min…

Do you mint one new NFT per site? Then you impose an excessive burden on users per new site they visit.

Do you mint one NFT per address? If blocking only applies to the one site, a malicious operator can just spam the next site using that address - after all, they own tons of addresses and have many sites to spam, and they can surely spam for at least a bit before getting caught (per site and per address).

If blocking is a public operation that gets you banned everywhere, well now one callous server owner can now disable your address’s ability to access anything.

I fail to see how Web3 NFTs solve any of these problems…

Post reply on HN