Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Miasma: A tool to trap AI web scrapers in an endless poison pit

github.com

Post reply on HN
Print view View original

11–20 of 276 posts

Re: Miasma: A tool to trap AI web scrapers in an endless poison pit

#11

Post by 20k »

sd9 wrote: post #9
madeofpalk wrote: post #8

Is there any evidence or hints that these actually work? It seems pretty reasonable that any scraper would already have mitigations for things like this as a function of just being on the internet.

Even it did work, I just can't bring myself to care enough. It doesn't feel like anything I could do on my site would make any material difference. I'm tired.

I definitely get this. The thing that gives me hope is that you only need to poison a very small % of content to damage AI models pretty significantly. It helps combat the mass scraping, because a significant chunk of the data they get will be useless, and its very difficult to filter it by hand

Re: Miasma: A tool to trap AI web scrapers in an endless poison pit

#14

Post by nubg »

madeofpalk wrote: post #8

Is there any evidence or hints that these actually work? It seems pretty reasonable that any scraper would already have mitigations for things like this as a function of just being on the internet.

What kind of migitations? How would you detect the poison fountain?

Re: Miasma: A tool to trap AI web scrapers in an endless poison pit

#15

Post by avereveard »

nubg wrote: post #14
madeofpalk wrote: post #8

Is there any evidence or hints that these actually work? It seems pretty reasonable that any scraper would already have mitigations for things like this as a function of just being on the internet.

What kind of migitations? How would you detect the poison fountain?

style="display: none;" aria-hidden="true" tabindex="1"

many scraper already know not to follow these, as it's how site used to "cheat" pagerank serving keyword soups

Re: Miasma: A tool to trap AI web scrapers in an endless poison pit

#18

Post by GaggiX »

nubg wrote: post #14
madeofpalk wrote: post #8

Is there any evidence or hints that these actually work? It seems pretty reasonable that any scraper would already have mitigations for things like this as a function of just being on the internet.

What kind of migitations? How would you detect the poison fountain?

Because the internet is noisy and not up to date all recent LLMs are trained using Reinforcement Learning with Verifiable Rewards, if a model has learned the wrong signature of a function for example it would be apparent when executing the code.

Re: Miasma: A tool to trap AI web scrapers in an endless poison pit

#19

Post by phoronixrly »

madeofpalk wrote: post #8

Is there any evidence or hints that these actually work? It seems pretty reasonable that any scraper would already have mitigations for things like this as a function of just being on the internet.

It does work, on two levels:

1. Simple, cheap, easy-to-detect bots will scrape the poison, and feed links to expensive-to-run browser-based bots that you can't detect in any other way.

2. Once you see a browser visit a bullshit link, you insta-ban it, as you can now see that it is a bot because it has been poisoned with the bullshit data.

My personal preference is using iocaine for this purpose though, in order to protect the entire server as opposed to a single site.

Post reply on HN