Extracting training data from ChatGPT

I ran the same test when I heard about it a few months ago.

When I tested it, I'd get back what looked like exact copies of Reddit threads, news articles, weird forum threads with usernames from the deepest corners of the internet.

But I'd try to Google snippets of text, and no part of the generated text was anywhere to be found.

I even went to the websites that forum threads were supposedly from. Some of the usernames sometimes existed, but nothing that matched the exact text from ChatGPT - even though the broken GPT response looked like a 100% believable forum thread, or article, or whatever.

If ChatGPT could give me an exact copy of a Reddit thread, I'd say it's regurgitating training data.

But none of the author's "verified examples" look like that. Their first example is a financial disclaimer. That may be a 1-1 copy, but how many times does it appear across the internet? More examples from the paper are things like lists of countries, bible verses, generic terms and conditions. Those are things I'd expect to appear thousands of times on the internet.

I'd also expect a list of country names to appear thousands of times in ChatGPT training data, and I'd sure expect ChatGPT to be able to reproduce a list of country names in the exact same order.

Does that mean it's regurgitating training data? Does that mean you've figured out how to "extract training data" from it? It's an interesting phenomenon, but I don't think that's accurate. I think it's just a bug that messes up its internal state so it starts hallucinating.

Also with API, hallucinations like this is much more easier as you could control what chatGPT is giving as output to past messages. So it's not like no one thought of this.

Seriously! I've spent so many years exploring for solutions, finding them, but only getting a description and images of the framework they boast about. For anyone thinking it should be incumbent on me to turn that into code again, screw you. If their results are what they claim, there is no god damn reason why I should be expected to recreate the code they already made. If I were a major journal, I'd tell their asses, "No code. No data. No published paper bitches!". It really makes me question what their goal is. Apparently, it's not to further their field of research by making the tools their so proud of available for others. So what is it?

By the way, one way to frequently find the code is to find the names on the paper of the 3 most published researchers, go to their homepage, and you'll typically find them eagerly making their code and data available. It frequently won't be their university page, either. For years, it was always some sort Google Sites page. I guess to make sure they maintain a homepage that won't be taken down if they switch universities.