I wonder how OpenAI are going to avoid the problem after the web is littered with its content?
They probably fingerprint their generated content.
How could that possibly work?
Training open-source LLMs on ChatGPT output is a really bad idea.
11–20 of 78 posts
Re: Training open-source LLMs on ChatGPT output is a really bad idea.
#12Earlier quoted context omitted.
They probably fingerprint their generated content.
How could that possibly work?
Well, they have all of the outputs of ChatGPT stored on their own servers. I suppose it wouldn't be out of the question to filter any future datasets they scrape against the outputs they have.
Re: Training open-source LLMs on ChatGPT output is a really bad idea.
#13Re: Training open-source LLMs on ChatGPT output is a really bad idea.
#14Earlier quoted context omitted.
They probably fingerprint their generated content.
How could that possibly work?
Computerphile has a video that explains it very well: https://youtu.be/XZJc1p6RE78
(You can skip to the section “Verifying“)
Re: Training open-source LLMs on ChatGPT output is a really bad idea.
#15I wonder how OpenAI are going to avoid the problem after the web is littered with its content?
Presumably the ChatGPT content that makes it onto the web is at the very least curated by humans, making that text on average slightly higher quality than the raw output of ChatGPT. If that's the case than you would expect model performance to continue to improve even if the dataset is polluted.
That's a bold assumption. I can imagine a world where 99.999% of the web will be filled with non-human curated AI generated text.
The rate at which AI can generate text will be so much greater than what humans can generate.
Re: Training open-source LLMs on ChatGPT output is a really bad idea.
#16> So I can easily imagine a near future where the web will be flooded by LLM output or at least by content heavily inspired or edited by LLMs. To be fair, we're already there, and we've been there for at least 10 years now. I'd wager >75% of the internet is garbage: auto-generated blog posts, programmatically-permuted ads, YouTube videos that mainly regurgitate other sources. Email is mostly garbage and the only reas…
i agree, but this stuff could very likely be a huge force multiplier.
Re: Training open-source LLMs on ChatGPT output is a really bad idea.
#17> So I can easily imagine a near future where the web will be flooded by LLM output or at least by content heavily inspired or edited by LLMs. To be fair, we're already there, and we've been there for at least 10 years now. I'd wager >75% of the internet is garbage: auto-generated blog posts, programmatically-permuted ads, YouTube videos that mainly regurgitate other sources. Email is mostly garbage and the only reas…
Yeah, I remember reading about things like sports reports and weather being generated by computers ages ago in the likes of SciAm or New Scientist. I don't recall if they used the term AI, I think they did but this was a long time ago.
Re: Training open-source LLMs on ChatGPT output is a really bad idea.
#18Do yourself a favor and skip right through to the Twitter link to another link to this excellent post by Yoav Goldberg [1] on the actual reason that training new models on ChatGPT output in the manner of supervised learning (in contrast to reinforcement learning) will not produce a model as good as ChatGPT
>For this type of interaction, we must use RL training, as supervised training teaches the model to lie. The core issue is that we want to encourage the model to answer based on its internal knowledge, but we don't know what this internal knowledge contains. In supervised training, we present the model with a question and its correct answer, and train the model to replicate the provided answer.
The author says he’s summarizing a talk by John Schulman of OpenAI [2] but I haven’t personally watched the video. In any case, this is an interesting insight.
Say we set up a supervised learning scenario where we ask the model to use its internal knowledge to answer a question and compare its answer to one written by a human. If the two answers essentially say the same thing, but in different words, in the supervised learning case the model is penalized. In the RL case, it’s rewarded. That’s the difference.
1. https://gist.github.com/yoavg/6bff0fecd65950898eba1bb321cfbd...
Re: Training open-source LLMs on ChatGPT output is a really bad idea.
#19Earlier quoted context omitted.
Presumably the ChatGPT content that makes it onto the web is at the very least curated by humans, making that text on average slightly higher quality than the raw output of ChatGPT. If that's the case than you would expect model performance to continue to improve even if the dataset is polluted.
That's a bold assumption. I can imagine a world where 99.999% of the web will be filled with non-human curated AI generated text. The rate at which AI can generate text will be so much greater than what humans can generate.
Doesn't matter. We want high-quality text - it's not necessary for it to be human-written. Social signals like upvotes or PageRank will still remain useful even if most text is AI generated.
Re: Training open-source LLMs on ChatGPT output is a really bad idea.
#20how does one do this ? train an opensource LLM on chatgpt ? people have been talking about it so im intrigued.
is there a how to anywhere - not even sure which opensource model to use, etc