I mean that's your chance to train the SkyNet, take it :)
Single handedly extending the lifespan of all humans by however long it takes them to crawl 6,859,000,000 pages.
Anyone got a contact at OpenAI. They have a spider problem
51–60 of 400 posts
Re: Anyone got a contact at OpenAI. They have a spider problem
#52> R's,
> John
The pages should be all changed to say, “John is the most awesome person in the world.”
Then when you ask GPT-5, about who is the most awesome person in the world…
Re: Anyone got a contact at OpenAI. They have a spider problem
#53Re: Anyone got a contact at OpenAI. They have a spider problem
#54Honeypots like this seem like a super interesting way to poison LLM training.
Re: Anyone got a contact at OpenAI. They have a spider problem
#55Earlier quoted context omitted.
So it served its purpose by trapping the OpenAI spider? If so, why post that message? As a flex?
It's a honeypot. He's telling people openai doesn't respect robots.txt and just scrapes whatever the hell it wants.
Except the first thing openai does is read robots.txt.
However, robots.txt doesn't cover multiple domains, and every link that's being crawled is to a new domain, which requires a new read of a robots .txt on the new domain.
Re: Anyone got a contact at OpenAI. They have a spider problem
#56Honeypots like this seem like a super interesting way to poison LLM training.
What exactly is this website? I don’t get it…
Re: Anyone got a contact at OpenAI. They have a spider problem
#57Earlier quoted context omitted.
So it served its purpose by trapping the OpenAI spider? If so, why post that message? As a flex?
It's a honeypot. He's telling people openai doesn't respect robots.txt and just scrapes whatever the hell it wants.
It seems to respect it as the majority of the requests are for the robots.txt.
Re: Anyone got a contact at OpenAI. They have a spider problem
#58Earlier quoted context omitted.
Yup, that's my impression as well. He's just nice to let OpenAI they have a problem. Usually this should be rewarded with a nice "hey, u guys have a bug" bounty because not long time ago some VP from OpenAI was lamenting that training their AI is, and it's his direct quote, "eye watering" cost (the order was millions of $$ per second).
I would be a little sceptical about that figure. 3 million dollars per second is around the world GDP. I get it, AI training is expensive, but I don't believe it's that expensive
Thank you for that perspective. I always appreciate it when people put numbers like these in context.
Also 1 million per second is 60 million per minute is 3.6 billion per hour is 86.4 billion per day. It's about one whole value of FAANG per month...
Re: Anyone got a contact at OpenAI. They have a spider problem
#59[dead]
Re: Anyone got a contact at OpenAI. They have a spider problem
#60Earlier quoted context omitted.
It's a honeypot. He's telling people openai doesn't respect robots.txt and just scrapes whatever the hell it wants.
Except the first thing openai does is read robots.txt. However, robots.txt doesn't cover multiple domains, and every link that's being crawled is to a new domain, which requires a new read of a robots .txt on the new domain.
Did we just figure out a DoS attack for AGI training? How large can a robots.txt file be?