181–190 of 288 posts
Earlier quoted context omitted.
Ohai! I'm working on dataset poisoning. The early prototype generates vapid LinkedIn posts but future versions will be fully pluggable with WebAssembly.
Now I'm picturing an AI trained exclusively on LinkedIn posts. One could probably sell that model to an online ad agency for a pretty penny.
Worth noting that in general if you do any "is this Google or not" you should always check by IP address as there's many people spoofing the googlebot user agent.
https://developers.google.com/static/search/apis/ipranges/go...
Earlier quoted context omitted.
That sounds fun, I look forward to reading a writeup about that
So I can plan it, how much detail do you want? Here's what I have about the prototype: https://anubis.techaro.lol/docs/admin/honeypot/overview
What prevents AI scrapers from continuing to scrape sites that contain a tag but not follow the bad links?
I own a forum which currently has 23k online users, all of them bots. The last new post in that forum is from _2019_. Its topic is also very niche. Why are so many bots there? This site should have basically been scraped a million times by now, yet those bots seem to fetch the stuff live, on the fly? I don’t get it.
I love the insanity of this idea. Not saying it's a good idea, but it's a very highly entertaining one, and I like that! I've also had enormous luck with Anubis. AI scrapers found my personal Forgejo server and were hitting it on the order of 600K requests per day. After setting up Anubis, that dropped to about 100. Yes, some people are going to see an anime catgirl from time to time. Bummer. Reducing my fake traffic…
*anime jackalgirl Also you mentioned Anubis, so it’s creator will probably read this. Hi Xena!
Earlier quoted context omitted.
Ehm what would stop ai scrapers from using a browser like a normal user would? Google bot already does, it can execute js and can read spa client side generated content, so it proves can be done at scale, and I'm pretty sure some ai scrapers already do
rate limit per ip that progressively keeps decreasing req/mins every few mins?
Why is it so bad that AIs scrape your self-hosted blog?