It seems quite appropriate that luddites will opt out of the AI training space. Hopefully it will lead to better quality output.
The AI models would not exist without data the so called luddites created.
Go ahead and block AI web crawlers
21–24 of 24 posts
Re: Go ahead and block AI web crawlers
#22Earlier quoted context omitted.
After they ingested everything
We don't actually know what they trained on. If we want to trust them, they've always honored robots.txt, because why wouldn't they? On the other hand, if we want to believe they're as capricious as we want to make them out to be, they've got a copy of LibGen, and every single torrent out there to train it in on, on top of using robots.txt to find what you don't want crawled. robots.txt is great - it tells you where…
Re: Go ahead and block AI web crawlers
#23Earlier quoted context omitted.
The AI models would not exist without data the so called luddites created.
The best content is not created by luddites. Creative curiousity is generally a necessity for great art.
Re: Go ahead and block AI web crawlers
#24Earlier quoted context omitted.
We don't actually know what they trained on. If we want to trust them, they've always honored robots.txt, because why wouldn't they? On the other hand, if we want to believe they're as capricious as we want to make them out to be, they've got a copy of LibGen, and every single torrent out there to train it in on, on top of using robots.txt to find what you don't want crawled. robots.txt is great - it tells you where…
Gpt-4 was released to the public on March 14th last year. They told people how to block their agent in August.
User agent: *