ChatGPT produces made-up nonexistent references

I was trying to find references to mathematical treatment of lists, and queried ChatGPT the following:

Please provide references that deal with the mathematical properties of lists

To which ChatGPT responded as follows:

    "Introduction to the Mathematics of Lists" by Peter J. Cameron (https://www.cambridge.org/core/books/introduction-to-the-mathematics-of-lists/3F0C6D0E69E49BF6D5CD6CA5D6A761B2)

    "List Processing and Programming" by John F. Sowa (https://www.cs.cmu.edu/~fp/courses/lp/lectures.html)

    "List Processing in Prolog" by David H. D. Warren (https://www.cs.purdue.edu/homes/hosking/726/html/prolog_lists.html)

    "Introduction to the Mathematics of Lists and Trees" by Meinard Müller (https://link.springer.com/book/10.1007%2F978-3-319-05147-1)

    "Mathematics of Lists and Trees" by Markus N. Rabe (https://www.researchgate.net/publication/296955213_Mathematics_of_Lists_and_Trees)

I was pretty surprised and happy, because I hadn't had much success with Google. But it turns out that every single one of those references and links are made up. The references don't exist and the links themselves appear to be cobbled together. The last link for example, takes you to a paper on "Grain mixes for subgrade layers" - the title in the url is irrelevant and only the number matters.

Googling for some of those authors throws up author pages with their publications, and it turns out they've never published papers with those titles! See for example, http://www-groups.mcs.st-andrews.ac.uk/~pjc/publ/

I’ve seen gpt-3 do this in general, it’s quite interesting. It’ll quote things that sound right to the prompt and response with realistic references and names, with no actual results. These types of AI seem to be against admitting they don’t know.

Typing a list of names of contributors when you're writing documentation with Copilot on also gives you made up, out of context names.

I'm not really surprised it hallucinates URLs. It is a generative language model, after all...

I tried to ask it for a list of books about the urban history with their respective authors in a Table.

At first sight the result looked believable, but after looking them up, most of the books don't exist.

What was especially interesting though: For one of the books that does actually exist, it completely made up the name of the author. According to it "The City: A Global History" was written by "Thomas Kaufmann", when the author actually is Joel Kotkin.

When asking it about Thomas Kaufmann, it made up facts, like him being professor at the University of Zurich. After some research it turns out an author named "Thomas Kaufmann" does exist, but he is a professor on theology at the University of Oslo and did not write any books about urban history.

The best thing though: When I asked about books by Joel Kotkin it suddenly correctly gave me "The City: A Global History", but noted that it was co-authored by "Thomas Kaufmann".

It's an effect of it being trained on internet data. People generally don't post when they realize they don't know enough about a topic (or post total nonsense, but looks like the dataset team has been able to filter that out). There's almost no text on the internet where there's a high-quality analysis based on what the writer knows, while being clear about their knowledge gaps. Either they look it up and fill in the gaps before posting, or simply decide not to post.

Maybe high school and college students' closed-book exam answers could be added to the training dataset? That might help to some extent.