21–30 of 162 posts
GTP has always been the worlds best bullshit simulator. Its output is incredible until you look closely and realize that it’s not so much inaccurate, but completely hallucinated. It’s the kind of thing that would be great for filling in Lorem Ipsum placeholder text that’s just convincing enough if no one looks too close.
GTP has always been the worlds best bullshit simulator. Its output is incredible until you look closely and realize that it’s not so much inaccurate, but completely hallucinated. It’s the kind of thing that would be great for filling in Lorem Ipsum placeholder text that’s just convincing enough if no one looks too close.
It fooled you at first because that is what it is designed to do, all of these look probable, as if they could be real. But the model has no notion of an actual scientific paper, which is connected to its author and which is published somewhere. The whole model is uniquely unfit as a search engine, it doesn't have a structured index of knowledge and information, where it could retrieve the data you want from. It will just make up stuff and it actually suceeded at its goal, which was creating a believable result.
I’ve seen gpt-3 do this in general, it’s quite interesting. It’ll quote things that sound right to the prompt and response with realistic references and names, with no actual results. These types of AI seem to be against admitting they don’t know.
Why would anyone expect a language model to admit that it "doesn't know" (unless explicitly asked to)? That's not what it's for. It's there to put together a string of words that's plausibly looking like an answer for a given prompt - that it can sometimes successfully reach for facts it was trained on while making up the answer is an added bonus, a side-effect of how it works.
There are several resources that deal with the mathematical properties of lists, including:
"Lists and Sets: The Mathematics of Computer Science" by Keith H. Rosen, which provides a comprehensive overview of the mathematical concepts and principles that underlie the use of lists in computer science.
"Discrete Mathematics with Applications" by Susanna S. Epp, which includes a chapter on lists and their mathematical properties, including operations such as concatenation and reversal.
"Introduction to the Theory of Computation" by Michael Sipser, which discusses the use of lists in the context of automata and formal languages, including their role in the construction of Turing machines.
"Data Structures and Algorithms in Python" by Michael T. Goodrich et al., which provides a detailed discussion of the mathematical properties of lists, including their time and space complexity.
"A Course in Combinatorics" by J. H. van Lint and R. M. Wilson, which includes a chapter on set systems and their properties, including lists as a special case.
GTP has always been the worlds best bullshit simulator. Its output is incredible until you look closely and realize that it’s not so much inaccurate, but completely hallucinated. It’s the kind of thing that would be great for filling in Lorem Ipsum placeholder text that’s just convincing enough if no one looks too close.
The biggest problem is the internet doesn't usually look too close
I say this is unexpected because a few years ago most people expected AI models to solve mundane tasks like automating Excel spreadsheet, while artists were considered pretty safe. Now the opposite has come to pass. The artistic power of these models is amazing, but I would never trust them to do anything that requires precision.
The biggest problem these models have is not necessarily that they produce "incorrect" (in quotes because they are not actually incorrect, just not what it's trained on) outputs, but rather that they have no built-in mechanism to express uncertainty in a meaningful way that's calibrated for the end-task of producing correct information.
GTP has always been the worlds best bullshit simulator. Its output is incredible until you look closely and realize that it’s not so much inaccurate, but completely hallucinated. It’s the kind of thing that would be great for filling in Lorem Ipsum placeholder text that’s just convincing enough if no one looks too close.
That’s basically the principle of the Turing test.
But maybe the GPT approach is fundamentally incapable of withstanding very detailed scrutiny.
I started looking and it turns out that OpenAI is keenly aware of the hallucination problem, and even wrote about how they're correcting for it in this blog post about InstructGPT: https://openai.com/blog/instruction-following/
To summarize: it seems OpenAI is addressing this by taking human-labeled output data from GPT and feeding this into new models, using a reward function to train the future models to behave with less hallucination. This way of working struck me as a mix of software engineering and crop breeding.
They discuss the trade-offs involved with doing this. The "hallucination" effect is probably one of the features that make GPT so creative.