I Have Blocked OpenAI
gavinhoward.com
I Have Blocked OpenAI
1–10 of 57 posts
Re: I Have Blocked OpenAI
#2unpopular opinion: llm responses being wrong is still valuable to me since it gives me a better jumping off point to exploring than nothing at all. especially with something like coding that can easily be back-propagated due to something not compiling/not working as intended. could be harmful in other areas tho.
Re: I Have Blocked OpenAI
#3For the last two weeks my little webserver has been getting 200+ hits a day from bots with the useragent of anthropic-ai. At first it was what you'd expect, mirroring all the pdfs and such. But the last week it's just /robots.txt. 200+ times per day from amazon-ec2 so I have no way of knowing if it's actually anthropic-ai.
I was happy that they'd be including documents on topics I found interesting and things I wrote in the word adjacency training of their foundational model. That'd mean the model would be more useful to me. But the robots.txt stuff is weird. Maybe it's because I've had,
User-agent: Killer AI
Disallow: /~superkuh/
Re: I Have Blocked OpenAI
#4Just a thought that I have, wouldn’t it be better to block all robots and only to whitelist a select few? More AI bots are scraping now and in the future…
Re: I Have Blocked OpenAI
#5I bet information about his software is around elsewhere, and now ChatGPT will make up even more. I don't know how this is fixed. Structured queryable data, I guess.
Re: I Have Blocked OpenAI
#6Re: I Have Blocked OpenAI
#7I agree that LLMs are almost more likely than not to answer documentation questions wrong, to hallucinate methods that don’t exist, or just be silly. But the value I see in allowing LLMs to train on documentation is in the glue code that an LLM could (potentially!) generate.
Documentation, even good docs, usually only answer the question “What does this method/class/general idea do?” Really good docs will come with some examples of connecting A and B. But they will often not include examples of connecting A to E when you have to transform via P because of business requirements, and almost never tell you how to incorporate third-party libraries X, Y, and Z.
As an engineer, I can read the docs and figure out the bits, but having an LLM suggest some of the intermediary or glue steps, even if wrong sometimes, is a benefit I don’t get only from good documentation.
Re: I Have Blocked OpenAI
#8unpopular opinion: llm responses being wrong is still valuable to me since it gives me a better jumping off point to exploring than nothing at all. especially with something like coding that can easily be back-propagated due to something not compiling/not working as intended. could be harmful in other areas tho.
Author here.
My docs will include tutorial links at the top, and those tutorials will focus on accomplishing common tasks.
I believe that's a good jumping off point.
Re: I Have Blocked OpenAI
#9I bet information about his software is around elsewhere, and now ChatGPT will make up even more. I don't know how this is fixed. Structured queryable data, I guess.
Author here.
You are correct, but if I demonstrate that I have done what I could to deny OpenAI access, and they still have it in their model, then I probably have more legal recourse against them.
Re: I Have Blocked OpenAI
#10This is such a technopurist take. People who use LLM’s already know they can give wrong information. Your documentation won’t be able to cover every single possible contextual scenario that an LLM can help with. I think there are valid reasons to not allow OpenAI to spider you, but this is just a really silly one that feels pretty egotistical. People aren’t going to this guy saying “well OpenAI said your software works this way and it doesn’t”. It’s an entirely contrived scenario that doesn’t exist in reality.