...at some point, some people started appreciating mailing lists and the distributed nature of Git again.
And Usenet, and IRC with a registered user prereq to join.
Also, set AI tarpits as fake links with recursive calls. Make then mad with non-curated bullshit made from Markov chain generators until their cache begins to rot forever.
These are DDOS attacks and should be treated in law as such. (Although I do realise that in many countries now we no longer have any effective "rule of law")
In the world, the richest wins, not the nicest (cf. Sam Altman)
These are DDOS attacks and should be treated in law as such. (Although I do realise that in many countries now we no longer have any effective "rule of law")
At some point it's easier to geoblock a whole country at the firewall level and loginwall the rest of the world, rather than trying to explain that in your jurisdiction, which is not their jurisdiction, what they are doing is a crime — which they don't give a single fuck about.
Yep -- our story here: https://about.readthedocs.com/blog/2024/07/ai-crawlers-abuse... (quoted in the OP) -- everyone I know has a similar story who is running large internet infrastructure -- this post does a great job of rounding a bunch of them up in 1 place. I called it when I wrote it, they are just burning their goodwill to the ground. I will note that one of the main startups in the space worked with us direct…
> just burning their goodwill to the ground AI firms seem to be leading from a position that goodwill is irrelevant: a $100bn pile of capital, like an 800lb gorilla, does what it wants. AI will be incorporated into all products whether you like it or not; it will absorb all data whether you like it or not.
AI tarpits && lim (human curated contant/mediocre AI answers -> 0) = AI's crumbling into dust by themselves.
Across my sites -- mostly open data sites -- the top 10 referrers are all bots. That doesn't include the long tail of randomized user agents that we get from the Alibaba netblocks.
At this point, I think we're well under 1% actual users on a good day.
It's going to get to the point where everything will be put behind a login to prevent LLM scrapers scanning a site. Annoying but the only option I can think of. If they use an account for scraping you just ban the account.