Anyone got a contact at OpenAI. They have a spider problem

> Before someone tells me to fix my robots.txt, this is a content farm so rather than being one web site with 6,859,000,000 pages, it is 6,859,000,000 web sites each with one page.

Re: Anyone got a contact at OpenAI. They have a spider problem

#8

Post by ajuc » Thu, Apr 11, 2024, 2:08 PM UTC

I mean that's your chance to train the SkyNet, take it :)

Re: Anyone got a contact at OpenAI. They have a spider problem

#9

Post by mypastself » Thu, Apr 11, 2024, 2:08 PM UTC

Frankly, I didn’t get the purpose of the website at first either. I guess I have an arachnid intellect.

Re: Anyone got a contact at OpenAI. They have a spider problem

#10

Post by fl7305 » Thu, Apr 11, 2024, 2:08 PM UTC

egberts1 wrote: post #5

There is always IP filtering, DNS blocking, and HTTP agent screening. Just sayin'.

I think he's saying that it's not a problem for him, but for OpenAI?