241–250 of 400 posts
Earlier quoted context omitted.
LLMs work at all because of the high correlation between the statistically most likely response and the most reasonable answer.
That's an explanation of why their answers can be useful, but doesn't relate to their ability to "not know" an answer
Earlier quoted context omitted.
It seems to respect it as the majority of the requests are for the robots.txt.
He says 3 million, and 1.8 million are for robots.txt So 1.2 million non robots.txt requests, when his robots.txt file is configured as follows # buzz off User-agent: GPTBot Disallow: / Theoretically if they were actually respecting robots.txt they wouldn't crawl any pages on the site. Which would also mean they wouldn't be following any links... aka not finding the N subdomains.
# silly bing
#User-agent: Amazonbot
#Disallow: /
# buzz off
#User-agent: GPTBot
#Disallow: /
# Don't Allow everyone
User-agent: *
Disallow: /archive
# slow down, dudes
#Crawl-delay: 60
Earlier quoted context omitted.
Except the first thing openai does is read robots.txt. However, robots.txt doesn't cover multiple domains, and every link that's being crawled is to a new domain, which requires a new read of a robots .txt on the new domain.
And they do have (the same) robots.txt on every domain, tailored for GPTbot, i.e. https://petra-cody-carlene.web.sp.am/robots.txt So, GPTBot is not following robots.txt, apparently.
Earlier quoted context omitted.
I wonder how much the source content is the cause of hallucinations rather than anything inherent to LLMs. I mean if someone posts a question on an internet forum that I don't know the answer to, I'm certainly not going to post "I don't know" since that wouldn't be useful. In fact, in general, in any non one-on-one conversation the answer "I don't know" is not useful because if you don't know in a group, your silence…
I'm really cross that the word "hallucination" has taken off to describe this as it's clearly in incorrect word. The correct word to describe it is "confabulation", which is clinically more accurate and a much clearer descriptor of what's actually going on. https://en.wikipedia.org/wiki/Confabulation
I’d let them do their thing, why not?! They want the internet? This is the real internet. It looks like he doesn’t really care that much that they’re retrieving millions of pages, so let them do their thing…
The CTO isn't even aware of where the data is coming from (allegedly).
Earlier quoted context omitted.
That’s the whole point. The site owner doesn’t want their information included in ChatGPT—they want you going to their website to view it instead. It’s functioning exactly as designed.
If my web browser's extension "visits" the site and dumps it into ChatGPT for me to read its summarization of the site, what has been gained by the website operator?
Earlier quoted context omitted.
Content you’re allowed and capable of scraping on the Internet is such a small amount of data, not sure why people are acting otherwise. Common crawl alone is only a few hundred TB, I have more content than that on a NAS sitting in my office that I built for a few grand (Granted I’m a bit of a data hoarder). The fears that we have “used all the data” are incredibly unfounded.
Gonna say you’re way off there. Once you decompress common crawl and index it for FTS and put it on fast storage you’re in for some serious pain, and that’s before you even put it in your ML pipeline. Even refined web runs about 2TB once loaded into Postgres with TS vector columns, and that’s a substantially smaller dataset than common crawl. It’s not just a dumping a to of zip files on your NAS, it’s making the data…
Earlier quoted context omitted.
AI training on AI-generated content is a future problem. Using textbooks is a good idea, until our textbooks are being written by AI. This problem can't really be avoided once we begin using AI to write, understand, explain, and disseminate information for us. It'll be writing more than blogs and SEO pages. How long before we start readily using AI to write academic journals and scientific papers? It's really only a…
You need to separate “content” and “knowledge.” GenAI can create massive amounts of content, but the knowledge you give it to create that content is what matters and why RAG is the most important pattern right now. From “known good” sources of knowledge, we can generate an infinite amount of content. We can add more “known good” knowledge to the model by generating content about that knowledge and training on it. I a…
That's my entire point -- AI only generates content right now, but it will also be the source of content for training purposes soon. We need a "known good" human knowledge-base, otherwise generative AI will degenerate as AI generated content proliferates.
Crawling the web, like in the case of the OP, isn't going to work for much longer. And books, video, and music are next.
Earlier quoted context omitted.
I wonder how much the source content is the cause of hallucinations rather than anything inherent to LLMs. I mean if someone posts a question on an internet forum that I don't know the answer to, I'm certainly not going to post "I don't know" since that wouldn't be useful. In fact, in general, in any non one-on-one conversation the answer "I don't know" is not useful because if you don't know in a group, your silence…
I'm really cross that the word "hallucination" has taken off to describe this as it's clearly in incorrect word. The correct word to describe it is "confabulation", which is clinically more accurate and a much clearer descriptor of what's actually going on. https://en.wikipedia.org/wiki/Confabulation