Earlier quoted context omitted.
[flagged]
No normal person has a chance against the capacity of a company like Facebook
Anyone can send 10k concurrent requests with no more than their mobile phone.
AI companies cause most of traffic on forums
141–150 of 466 posts
Re: AI companies cause most of traffic on forums
#142Earlier quoted context omitted.
Unless they're a big company in which case they can DMCA anything they want, and they get the benefit of the doubt.
Can you even DMCS takedown crawlers?
Doubt it, a vanilla cease-and-desist letter would probably be the approach there. I doubt any large AI company would pay attention though, since, even if they're in the wrong, they can outspend almost anyone in court.
Re: AI companies cause most of traffic on forums
#143I have a hypothetical question: lets say I want to slightly scramble the content of my site (no so much so as to be obvious, but enough that most knowledge within is lost) when I detect that a request is coming from one of these bots, could I face legal repercussions?
Re: AI companies cause most of traffic on forums
#144Earlier quoted context omitted.
[flagged]
Can't every webserver crash due to being overloaded? There's an upper limit to performance of everything. My website is a hobby and has a budget of $4/mo budget VPS. Perhaps I'm saying crash and you're interpreting that as a bug but really it's just an OOM issue cause of too many in-flight requests. IDK, I don't care enough to handle serving my website at Facebook's scale.
I suspect if the tables were turned and someone managed to crash FB consistently they might not take too kindly to that.
Re: AI companies cause most of traffic on forums
#145Besides playing an endless game of wackamole by blocking the bots. What can we do?
I don’t see court system being helpful in recovering lost time. But maybe we could waste their time by fingerprinting the bot traffic and returning back useless/irrelevant content.
Re: AI companies cause most of traffic on forums
#146OpenAI publishes IP ranges for their bots, https://github.com/greyhat-academy/lists.d/blob/main/scraper... For antisocial scrapers, there's a Wordpress plugin, https://kevinfreitas.net/tools-experiments/ > The words you write and publish on your website are yours. Instead of blocking AI/LLM scraper bots from stealing your stuff why not poison them with garbage content instead? This plugin scrambles the words in the c…
> OpenAI publishes IP ranges for their bots... If blocking them becomes standard practice, how long do you think it'd be before they started employing third-party crawling contractors to get data sets?
[deleted]
Re: AI companies cause most of traffic on forums
#147What a disgrace... I am appalled: Not only are they intent on ruin incomes and jobs. They are not even good net citizens.
This needs to stop. Assuming free services have pools of money; many are funded by good people that provide a safe place.
Many of these forums are really important and are intended for humans to get help and find people like them etc.
There has to be a point soon where action and regulation is needed. This is getting out of hand.
Re: AI companies cause most of traffic on forums
#148AI companies go on forums to scrape content for training models, which are surreptitiously used to generate content posted on forums, from which AI companies scrape content to train models, which are surreptitiously used to generate content posted on forums... It's a lot of traffic, and a lot of new content, most of which seems to add no value. Sigh.
Re: AI companies cause most of traffic on forums
#149Their appetite cannot be quenched, and there is little to no value in giving them access to the content. I have data... 7d from a single platform with about 30 forums on this instance. 4.8M hits from Claude 390k from Amazon 261k from Data For SEO 148k from Chat GPT That Claude one! Wowser. Bots that match this (which is also the list I block on some other forums that are fully private by default): (?i). (AhrefsBot|AI…
You just plain blocking anyone using node from programatically accessing your content with Axios?