Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: Stop AI scrapers from hammering your self-hosted blog (using porn)

github.com

Post reply on HN
Print view View original

101–110 of 288 posts

Re: Show HN: Stop AI scrapers from hammering your self-hosted blog (using porn)

#101

Post by ddtaylor »

eek2121 wrote: post #92

Disclosure, I've not run a website since my health issues began, however, Cloudflare has an AI firewall, Cloudflare is super cheap (also: unsure if the AI firewall is on the free tier, however I would be surprised if it is not). Ignoring the recent drama about a couple incidents they've had (because this would not matter for a personal blog), why not use this instead? Just curious. Hoping to be able to work on a webs…

Cloudflare has created a bit of grief with regular users getting spammed with "prove your human" requests.

Re: Show HN: Stop AI scrapers from hammering your self-hosted blog (using porn)

#102

Post by n1xis10t »

xena wrote: post #99
n1xis10t wrote: post #13

Earlier quoted context omitted.

*anime jackalgirl Also you mentioned Anubis, so it’s creator will probably read this. Hi Xena!

Ohai! I'm working on dataset poisoning. The early prototype generates vapid LinkedIn posts but future versions will be fully pluggable with WebAssembly.

That sounds fun, I look forward to reading a writeup about that

Re: Show HN: Stop AI scrapers from hammering your self-hosted blog (using porn)

#103

Post by buu700 »

kstrauser wrote: post #11

I love the insanity of this idea. Not saying it's a good idea, but it's a very highly entertaining one, and I like that! I've also had enormous luck with Anubis. AI scrapers found my personal Forgejo server and were hitting it on the order of 600K requests per day. After setting up Anubis, that dropped to about 100. Yes, some people are going to see an anime catgirl from time to time. Bummer. Reducing my fake traffic…

It's actually a well established concept: https://youtu.be/p9KeopXHcf8

Re: Show HN: Stop AI scrapers from hammering your self-hosted blog (using porn)

#104

Post by username223 »

yupyupyups wrote: post #47
username223 wrote: post #7

The more ways people mess with scrapers, the better -- let a thousand flowers bloom! You as an individual can't compete with VC-funded looters, but there aren't enough of them to defeat a thousand people resisting in different ways.

We need to find more ways to poison their data.

> Wee knead two fine-d Moore Waze too Poisson there date... uh.

Yes. Revel in your creativity mocking and blocking the slop machines. The "remote refactor" command, "rm -rf", is the best way to reduce the cyclomatic complexity of a local codebase.

Re: Show HN: Stop AI scrapers from hammering your self-hosted blog (using porn)

#105

Post by n1xis10t »

tokioyoyo wrote: post #98
n1xis10t wrote: post #95

Earlier quoted context omitted.

If people start making search engines again and there is more competition for Google, I think things would be pretty sweet.

Because of the financial incentives, it would still end up with people doing things to drive traffic to their website though, no? Maybe because the web was smaller, and people looked at it as means "to explore curiosity" in the olden days it kinda worked differently... maybe I just got old, but I don't want to believe that.

By “doing things to drive traffic to their website” do you mean trying to do SEO type things to manipulate search engine rankings? If so, I think that there are probably ways to rank that are immune to tampering.

Don’t worry, you’re not just old. The internet kind of sucks now.

Re: Show HN: Stop AI scrapers from hammering your self-hosted blog (using porn)

#106

Post by n1xis10t »

username223 wrote: post #104
yupyupyups wrote: post #47

Earlier quoted context omitted.

We need to find more ways to poison their data.

> Wee knead two fine-d Moore Waze too Poisson there date... uh. Yes. Revel in your creativity mocking and blocking the slop machines. The "remote refactor" command, "rm -rf", is the best way to reduce the cyclomatic complexity of a local codebase.

Indeed, complexity (both cyclomatic and post-frontal) must be reduced such that the two spurving bearings make a direct line with the panametric fan.

For more details consult this instructional video: https://youtu.be/RXJKdh1KZ0w

Re: Show HN: Stop AI scrapers from hammering your self-hosted blog (using porn)

#107

Post by owl57 »

n1xis10t wrote: post #33
throawayonthe wrote: post #30

Earlier quoted context omitted.

i think making the case of "you are acting (sending web requests) while knowingly identifying as another legal entity (and criminally/libelously/etc)" shouldn't be toooo hard

Seems like, but there are tons of things that forge request headers all the time, and I don’t think I’ve heard of anyone getting in legal trouble for it. Now I think most of these are scrapers pretending to be browsers, so it might be different I don’t know.

And most of them are pretending to be Chrome. If Google had a good case against someone reusing their user agent, maybe they would already have sued?

Or maybe not. Got some random bot from my server logs. Yeah, it's pretending to be Chrome, but more exactly:

"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/131.0.0.0 Safari/537.36"

I guess Google might be not eager to open this can of worms.

Re: Show HN: Stop AI scrapers from hammering your self-hosted blog (using porn)

#108

Post by xena »

n1xis10t wrote: post #102
xena wrote: post #99

Earlier quoted context omitted.

Ohai! I'm working on dataset poisoning. The early prototype generates vapid LinkedIn posts but future versions will be fully pluggable with WebAssembly.

That sounds fun, I look forward to reading a writeup about that

So I can plan it, how much detail do you want? Here's what I have about the prototype: https://anubis.techaro.lol/docs/admin/honeypot/overview

Re: Show HN: Stop AI scrapers from hammering your self-hosted blog (using porn)

#109

Post by nkurz »

I was told by the admin of one forum site I use that the vast majority of the AI scraping traffic is Chinese at this point. Not hidden or proxied, but straight from China. Can anyone else confirm this?

Anyway, if it is true, and assuming a forum with minimal genuine Chinese traffic, might a simple approach that injects the porn links only into IP's accessing from China work?

Re: Show HN: Stop AI scrapers from hammering your self-hosted blog (using porn)

#110

Post by n1xis10t »

xena wrote: post #108
n1xis10t wrote: post #102

Earlier quoted context omitted.

That sounds fun, I look forward to reading a writeup about that

So I can plan it, how much detail do you want? Here's what I have about the prototype: https://anubis.techaro.lol/docs/admin/honeypot/overview

Probably any detail that you think is cool, I would be interested in reading about. When in doubt err on the side of too much detail.

That was a good read. I hadn’t heard of spintax before, but I’ve thought of doing things like that. Also “pseudoprofound anti-content”, what a great term, that’s hilarious!

Post reply on HN