11–20 of 48 posts
Earlier quoted context omitted.
I imagine it's everything on the web that's been crawled by Bing and hasn't blocked ChatGPT explicitly in its robots.txt file.
have openai's crawlers really paid attention to people's robots.txts?
Earlier quoted context omitted.
I imagine it's everything on the web that's been crawled by Bing and hasn't blocked ChatGPT explicitly in its robots.txt file.
have openai's crawlers really paid attention to people's robots.txts?
edit: enable in settings, log out+in, mouse over "GPT-4" at the top Hm, is this available in the web app?
Edit: Ah, never mind, I'm remembering wrong. They turned it off previously because it could bypass paywalls, not because of security concerns. So the answer is likely, "yes it is still just as insecure as ever, but now there are probably some checks to keep it from summarizing paywalled news articles." :shrug:
Earlier quoted context omitted.
have openai's crawlers really paid attention to people's robots.txts?
Yes. There's even a project tracking which newspapers are blocking them here: https://palewi.re/docs/news-homepages/openai-gptbot-robotstx...
It’s interesting that OpenAI themselves are not calling this “you can browse again” and also throwing in the “no longer limited by September 2021 cutoff” even though it is January 2022. Stinks to me like they are pulling a little marketing campaign to lure in new users and the media is happy to oblige to use the same language/phrasing. For reference, they disabled browsing because you could bypass paywalls with it.
I've been able to do that for over 25 years.
edit: enable in settings, log out+in, mouse over "GPT-4" at the top Hm, is this available in the web app?
I've been able to do that for over 25 years.