Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Feed the bots

maurycyz.com

Post reply on HN
Print view View original

41–50 of 216 posts

Re: Feed the bots

#41

Post by markus_zhang »

I have always recommended this strategy: flood the AI bots with garbage that looks like authentic information so that they need actual humans to filter the information. Make sure that every site does this so they get more garbage than real stuffs. Hike up the proportion so that even ordinary people eventually figure out that using these AI products has more harm than use because it just produces garbage. I just don't know what is the cost, now it looks like pretty doable.

If you can't fight them, flood them. If they want to open a window, pull down the whole house.

Re: Feed the bots

#42

Post by xyzal »

I think random text can be detected and filtered. We need probably pre-generated bad information to make utility of crawling one's site truly negative.

On my site, I serve them a subset of Emergent Misalignment dataset, randomly perturbed by substituting some words with synonyms.

It should make the LLMs trained on it behave like dicks according to this research https://www.emergent-misalignment.com/

Re: Feed the bots

#43

Post by isoprophlex »

fainpul wrote: post #2

This follow-up post has the details of the "Markov babbler": https://maurycyz.com/projects/trap_bots/

Very elegant and surprisingly performant. I hope the llm bros have a hard time cleaning this shit out of their scrapes.

Re: Feed the bots

#44

Post by krzyk »

But why?

Do they do any harm? They do provide source for material if users asks for it. (I frequently do because I don't trust them, so I check sources).

You still need to pay for the traffic, and serving static content (like text on that website) is way less CPU/disk expensive than generating anything.

Re: Feed the bots

#45

Post by peterlk »

markus_zhang wrote: post #41

I have always recommended this strategy: flood the AI bots with garbage that looks like authentic information so that they need actual humans to filter the information. Make sure that every site does this so they get more garbage than real stuffs. Hike up the proportion so that even ordinary people eventually figure out that using these AI products has more harm than use because it just produces garbage. I just don't…

LLMs can now detect garbage much more cheaply than humans can. This might increase cost slightly for the companies that own the AIs, but it almost certainly will not result in hiring human reviewers

Re: Feed the bots

#46

Post by blibble »

if you want to be really sneaky make it so the web doesn't start off infinite

because as infinite site that has appeared out of nowhere will quickly be noticed and blocked

start it off small, and grow it by a few pages every day

and the existing pages should stay 99% the same between crawls to gain reputation

Re: Feed the bots

#47

Post by kaoD »

krzyk wrote: post #44

But why? Do they do any harm? They do provide source for material if users asks for it. (I frequently do because I don't trust them, so I check sources). You still need to pay for the traffic, and serving static content (like text on that website) is way less CPU/disk expensive than generating anything.

What you're referring to are LLMs visiting your page via tool use. That's a drop in the ocean of crawlers that are racing to slurp as much of the internet as possible before it dries.

Re: Feed the bots

#48

Post by 8organicbits »

stubish wrote: post #28
OutOfHere wrote: post #17

The user's approach would work only if bots can accurately even be classified, but this is impossible. The end result is that the action is user's site is now nothing but markov garbage. Not only will bots desert it but humans will too.

The traditional approach is a link to the tarpit that the bots can see but humans can't, say using CSS to render it 0 pixels in size.

Please keep in mind that not all humans interact with web pages by "seeing". If you fool a scraper you may also fool someone using a screen reader.

Re: Feed the bots

#49

Post by hn8726 »

CaptainOfCoit wrote: post #14
8organicbits wrote: post #7

Earlier quoted context omitted.

The technical side is straightforward but the legal implications of trying passwords to try to scrape content behind authentication could pose a barrier. Using credentials that aren't yours, even if they are publicly known, is (in many jurisdictions) a crime. Doing it at scale as part of a company would be quite risky.

> but the legal implications of trying passwords to try to scrape content behind authentication could pose a barrier If you're doing something alike to cracking then yeah. But if the credentials are right there on the landing page, and visible to the public, it's not really cracking anymore since you already know the right password before you try it, and the website that put up the basic auth is freely sharing the pa…

Otoh if, as a human, you use a known (even leaked on the website) password to "bypass the security" in order to "gain access to content you're not authorized to see", I think you'd get in trouble. I'd like if the same logic aplied to bots - implement basic (albeit weak) security and only allow access to humans. This way bots have to _hack you_ to read the content

Re: Feed the bots

#50

Post by [deleted] »

krzyk wrote: post #44

But why? Do they do any harm? They do provide source for material if users asks for it. (I frequently do because I don't trust them, so I check sources). You still need to pay for the traffic, and serving static content (like text on that website) is way less CPU/disk expensive than generating anything.

[deleted]
Post reply on HN