Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AI companies cause most of traffic on forums

pod.geraspora.de

Post reply on HN
Print view View original

71–80 of 466 posts

Re: AI companies cause most of traffic on forums

#71

Post by jahewson »

CoastalCoder wrote: post #48
markerz wrote: post #23

One of my websites was absolutely destroyed by Meta's AI bot: Meta-ExternalAgent https://developers.facebook.com/docs/sharing/webmasters/web-... It seems a bit naive for some reason and doesn't do performance back-off the way I would expect from Google Bot. It just kept repeatedly requesting more and more until my server crashed, then it would back off for a minute and then request more again. My solution was to add…

I wonder if it would work to send Meta's legal department a notice that they are not permitted to access your website. Would that make subsequent accesses be violations of the U.S.'s Computer Fraud and Abuse Act?

No, fortunately random hosts on the internet don’t get to write a letter and make something a crime.

Re: AI companies cause most of traffic on forums

#72

Post by walterbell »

GaggiX wrote: post #46
walterbell wrote: post #29

Earlier quoted context omitted.

Attackers don't have a monopoly on LLM expertise, defenders can also use LLMs for obfuscation. Technology arms races are well understood.

I hate LLM companies, I guess I'm going to use OpenAI API to "obfuscate" the content or maybe I will buy an NVIDIA GPU to run a llama model, mhm maybe on GPU cloud.

With tiny amounts of forum text, obfuscation can be done locally with open models and local inference hardware (NPU on Arm SoC). Zero dollars sent to OpenAI, NVIDIA, AMD or GPU clouds.

Re: AI companies cause most of traffic on forums

#73

Post by throwaway_fai »

skywhopper wrote: post #8
jeffbee wrote: post #5

Ironic that there is a dichotomy between Google and Bing with orders of magnitude less traffic than AI organizations, because only Google really has fresh docs. Bing isn't terrible but their index is usually days old. But something like Claude is years out of date. Why do they need to crawl that much?

They don’t. They are wasting their resources and other people’s resources because at the moment they have essentially unlimited cash to burn burn burn.

Keep in mind too, for a lot of people pushing this stuff, there's an essentially religious motivation that's more important to them than money. They truly think it's incumbent on them to build God in the form of an AI superintelligence, and they truly think that's where this path leads.

Yet another reminder that there are plenty of very smart people who are, simultaneously, very stupid.

Re: AI companies cause most of traffic on forums

#75

Post by [deleted] »

GaggiX wrote: post #51
botanical76 wrote: post #47

Earlier quoted context omitted.

You're right, this approach is too easy to spot. Instead, pass all your blog posts through an LLM to automatically inject grammatically sound inaccuracies.

Are you going to use OpenAI API or maybe setup a Meta model on an NVIDIA GPU? Ahah Edit: I found it funny to buy hardware/compute to only fund what you are trying to stop.

[deleted]

Re: AI companies cause most of traffic on forums

#76

Post by acedTrex »

azemetre wrote: post #70
betaby wrote: post #58

Earlier quoted context omitted.

Crashing wasn't the intent. And scraping is legal, as I remember per Linkedin case.

There’s a fine line between scrapping and DDOS’ing I’m sure. Just because you manufacture chemicals doesn’t mean you can legally dump your toxic waste anywhere you want (well shouldn’t be allowed to at least). You also shouldn’t be able to set your crawlers causing sites to fail.

intent is likely very important to something like a ddos charge

Re: AI companies cause most of traffic on forums

#77

Post by botanical76 »

GaggiX wrote: post #51
botanical76 wrote: post #47

Earlier quoted context omitted.

You're right, this approach is too easy to spot. Instead, pass all your blog posts through an LLM to automatically inject grammatically sound inaccuracies.

Are you going to use OpenAI API or maybe setup a Meta model on an NVIDIA GPU? Ahah Edit: I found it funny to buy hardware/compute to only fund what you are trying to stop.

I suppose you are making a point about hypocrisy. Yes, I use GenAI products. No, I do not agree with how they have been trained. There is nothing individuals can do about the moral crimes of huge companies. It's not like refusing to use a free Meta LLama model constitutes as voting with your dollars.

Re: AI companies cause most of traffic on forums

#78

Post by jsheard »

devit wrote: post #63
markerz wrote: post #23

One of my websites was absolutely destroyed by Meta's AI bot: Meta-ExternalAgent https://developers.facebook.com/docs/sharing/webmasters/web-... It seems a bit naive for some reason and doesn't do performance back-off the way I would expect from Google Bot. It just kept repeatedly requesting more and more until my server crashed, then it would back off for a minute and then request more again. My solution was to add…

[flagged]

That's right, getting DDOSed is a skill issue. Just have infinite capacity.

Re: AI companies cause most of traffic on forums

#79

Post by GaggiX »

walterbell wrote: post #72
GaggiX wrote: post #46

Earlier quoted context omitted.

I hate LLM companies, I guess I'm going to use OpenAI API to "obfuscate" the content or maybe I will buy an NVIDIA GPU to run a llama model, mhm maybe on GPU cloud.

With tiny amounts of forum text, obfuscation can be done locally with open models and local inference hardware (NPU on Arm SoC). Zero dollars sent to OpenAI, NVIDIA, AMD or GPU clouds.

>local inference hardware (NPU on Arm SoC).

Okay the battle is already lost from the beginning.

Re: AI companies cause most of traffic on forums

#80

Post by frereubu »

coldpie wrote: post #65
buro9 wrote: post #56

Their appetite cannot be quenched, and there is little to no value in giving them access to the content. I have data... 7d from a single platform with about 30 forums on this instance. 4.8M hits from Claude 390k from Amazon 261k from Data For SEO 148k from Chat GPT That Claude one! Wowser. Bots that match this (which is also the list I block on some other forums that are fully private by default): (?i). (AhrefsBot|AI…

You know, at this point, I wonder if an allowlist would work better.

I love (hate) the idea of a site where you need to send a personal email to the webmaster to be whitelisted.
Post reply on HN