Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AI companies cause most of traffic on forums

pod.geraspora.de

Post reply on HN
Print view View original

181–190 of 466 posts

Re: AI companies cause most of traffic on forums

#181

Post by calibas »

jhull wrote: post #124

> And the best thing of all: they crawl the stupidest pages possible. Recently, both ChatGPT and Amazon were - at the same time - crawling the entire edit history of the wiki. And I mean that - they indexed every single diff on every page for every change ever made. Is it stupid? It makes sense to scrape all these pages and learn the edits and corrections that people make.

It seems like they just grabbing every possible bit of data available, I doubt there's any mechanism to flag which edits are corrections when training.

Re: AI companies cause most of traffic on forums

#182

Post by alehlopeh »

uludag wrote: post #103

I'm always curious how poisoning attacks could work. Like, suppose that you were able to get enough human users to produce poisoned content. This poisoned content would be human written and not just garbage, and would contain flawed reasoning, misjudgments, lapses of reasoning, unrealistic premises, etc. Like, I've asked ChatGPT certain questions where I know the online sources are limited and it would seem that from…

Sorry but you’re assuming that “real” content is devoid of flawed reasoning, misjudgments, etc?

Re: AI companies cause most of traffic on forums

#184

Post by binarymax »

> If you try to rate-limit them, they’ll just switch to other IPs all the time. If you try to block them by User Agent string, they’ll just switch to a non-bot UA string (no, really). This is literally a DDoS on the entire internet.

I am of the opinion that when an actor is this bad, then the best block mechanism is to just serve 200 with absolute garbage content, and let them sort it out.

Re: AI companies cause most of traffic on forums

#185

Post by gashad »

What sort of effort would it take to make an LLM training honeypot resulting in LLMs reliably spewing nonsense? Similar to the way Google once defined the search term "Santorum"?

https://en.wikipedia.org/wiki/Campaign_for_the_neologism_%22... where

The way LLMs are trained with such a huge corpus of data, would it even be possible for a single entity to do this?

Re: AI companies cause most of traffic on forums

#186

Post by sangnoir »

GaggiX wrote: post #26
walterbell wrote: post #10

OpenAI publishes IP ranges for their bots, https://github.com/greyhat-academy/lists.d/blob/main/scraper... For antisocial scrapers, there's a Wordpress plugin, https://kevinfreitas.net/tools-experiments/ > The words you write and publish on your website are yours. Instead of blocking AI/LLM scraper bots from stealing your stuff why not poison them with garbage content instead? This plugin scrambles the words in the c…

I imagine these companies today are curing their data with LLMs, this stuff isn't going to do anything.

> I imagine these companies today are curing their data with LLMs, this stuff isn't going to do anything

The same LLMs tag are terrible at AI-generated-content detection? Randomly mangling words may be a trivially detectable strategy, so one should serve AI-scraper bots with LLM-generated doppelganger content instead. Even OpenAI gave up on its AI detection product

Re: AI companies cause most of traffic on forums

#187

Post by eesmith »

TuringNYC wrote: post #172
markerz wrote: post #23

One of my websites was absolutely destroyed by Meta's AI bot: Meta-ExternalAgent https://developers.facebook.com/docs/sharing/webmasters/web-... It seems a bit naive for some reason and doesn't do performance back-off the way I would expect from Google Bot. It just kept repeatedly requesting more and more until my server crashed, then it would back off for a minute and then request more again. My solution was to add…

>> One of my websites was absolutely destroyed by Meta's AI bot: Meta-ExternalAgent https://developers.facebook.com/docs/sharing/webmasters/web- ... Are they not respecting robots.txt?

Quoting the top-level link to geraspora.de:

> Oh, and of course, they don’t just crawl a page once and then move on. Oh, no, they come back every 6 hours because lol why not. They also don’t give a single flying fuck about robots.txt, because why should they. And the best thing of all: they crawl the stupidest pages possible. Recently, both ChatGPT and Amazon were - at the same time - crawling the entire edit history of the wiki.

Re: AI companies cause most of traffic on forums

#188

Post by bodantogat »

throwaway290 wrote: post #105
newsclues wrote: post #96

Earlier quoted context omitted.

The amateurs at home are going to give the big companies what they want: an excuse for government regulation.

If it doesn't say it's a bot and it doesn't come from a corporate IP it doesn't mean it's NOT a bot and not run by some "AI" company.

I have no way to verify this, I suspect these are either stealth AI companies or data collectors, who hope to sell training data to them

Re: AI companies cause most of traffic on forums

#189

Post by varelse »

llm_trw wrote: post #179
echelon wrote: post #129

Earlier quoted context omitted.

You could run all of your content through an LLM to create a twisted and purposely factually incorrect rendition of your data. Forward all AI bots to the junk copy. Everyone should start doing this. Once the AI companies engorge themselves on enough garbage and start to see a negative impact to their own products, they'll stop running up your traffic bills. Maybe you don't even need a full LLM. Just a simple transfor…

You will be burning through thousands of dollars worth of compute to do that.

[dead]

Re: AI companies cause most of traffic on forums

#190

Post by sangnoir »

pogue wrote: post #104
walterbell wrote: post #72

Earlier quoted context omitted.

With tiny amounts of forum text, obfuscation can be done locally with open models and local inference hardware (NPU on Arm SoC). Zero dollars sent to OpenAI, NVIDIA, AMD or GPU clouds.

What specifically are you suggesting? Is this a project that already exists or a theory of yours?

Markov chains are ancient in AI-years, and don't need a GPU.
Post reply on HN