GPTBot – OpenAI’s Web Crawler
platform.openai.com
GPTBot – OpenAI’s Web Crawler
1–10 of 327 posts
Re: GPTBot – OpenAI’s Web Crawler
#2Re: GPTBot – OpenAI’s Web Crawler
#3Re: GPTBot – OpenAI’s Web Crawler
#4Re: GPTBot – OpenAI’s Web Crawler
#5EDIT: scuba diving NOT scooba diving
Re: GPTBot – OpenAI’s Web Crawler
#6Nice of them to respect crawling after they've already trained their model. Presumably these headers don't affect any pages they've already crawled to train GPT(?)
Re: GPTBot – OpenAI’s Web Crawler
#7> For robots.txt, we do follow the same restrictions applied to googlebot, otherwise Google benefits from its dominant position.
https://community.brave.com/t/stop-website-being-shown-in-br...
Re: GPTBot – OpenAI’s Web Crawler
#8Good on them for following a standard interaction model.
Re: GPTBot – OpenAI’s Web Crawler
#9If you scrape my hobby website about photography, scuba diving or let's say baking or gardening which improves your model by let's say a delta of 0.00000000001 than shouldn't I get some free credits to use that model or proportionate share in the revenue stream? EDIT: scuba diving NOT scooba diving
openai is worth $29,000,000.00 you contributed 0.00000000001
punches numbers in calculator
thus the value of your free credits is 0.001 cents. minus any accounting fees.
Re: GPTBot – OpenAI’s Web Crawler
#10Also, where is this data going? The existing ChatGPT says it has nothing past 2021.