21–30 of 631 posts
First, it was Facebook https://news.ycombinator.com/item?id=23490367 and now it's these other companies.
What's worse? They completely ignore a simple HTTP 429 status.
Could one put a mangler on the responses to suspected bots to poison their data sets with nonsense code.. :/
...at some point, some people started appreciating mailing lists and the distributed nature of Git again.
And Usenet, and IRC with a registered user prereq to join. Also, set AI tarpits as fake links with recursive calls. Make then mad with non-curated bullshit made from Markov chain generators until their cache begins to rot forever.
This article starts by citing a blog article - displays a screenshot of the article - but doesn't link to it.
[1]: https://drewdevault.com/2025/03/17/2025-03-17-Stop-externali...
Could one put a mangler on the responses to suspected bots to poison their data sets with nonsense code.. :/
My question is, can we serve PoW challenges to these AI LLM scrapers that can be profitable?
That's what I've been doing! It works shockingly well. https://github.com/TecharoHQ/anubis
Perhaps time to start a central community ban pool for IP ranges?
Yep -- our story here: https://about.readthedocs.com/blog/2024/07/ai-crawlers-abuse... (quoted in the OP) -- everyone I know has a similar story who is running large internet infrastructure -- this post does a great job of rounding a bunch of them up in 1 place. I called it when I wrote it, they are just burning their goodwill to the ground. I will note that one of the main startups in the space worked with us direct…
> just burning their goodwill to the ground AI firms seem to be leading from a position that goodwill is irrelevant: a $100bn pile of capital, like an 800lb gorilla, does what it wants. AI will be incorporated into all products whether you like it or not; it will absorb all data whether you like it or not.
"Why should we care about open source maintainers" is just a microcosm of the much larger "why should we care about literally anybody" mindset.