Training open-source LLMs on ChatGPT output is a really bad idea.
1–10 of 78 posts
Re: Training open-source LLMs on ChatGPT output is a really bad idea.
#2The article points out that training data generated using ChatGPT is necessarily biased or tainted with the consequences of the policy optimizations and RLHF alignment processes conducted by OpenAI. This results in models that reflect the alignment preferences of OpenAI instead of the preferences of the model developers.
Re: Training open-source LLMs on ChatGPT output is a really bad idea.
#3It won’t work because will need as much training data as ChatGPT to get to its general knowledge level.
A subset will give you a subset of the knowledge, it’s no free lunch
Re: Training open-source LLMs on ChatGPT output is a really bad idea.
#4Seems to be a Multiplicity[1]-type problem.
Re: Training open-source LLMs on ChatGPT output is a really bad idea.
#5It won’t work because will need as much training data as ChatGPT to get to its general knowledge level. A subset will give you a subset of the knowledge, it’s no free lunch
The post is referring to fine tuning. You can train your model on the internet, but then it will just produce internet content and not behave subserviently (or carrying the operators desired polical biases rather than random internet ones). Models like ChatGPT take an internet trained model and then perform additional training to make it less internet-like and more cooperative.
Some recent research hash showed reasonable success transferring fine tuning between models using outputs.
Re: Training open-source LLMs on ChatGPT output is a really bad idea.
#6I wonder how OpenAI are going to avoid the problem after the web is littered with its content?
Re: Training open-source LLMs on ChatGPT output is a really bad idea.
#7I wonder how OpenAI are going to avoid the problem after the web is littered with its content?
what do we say to people who has the argument of "but the web is already littered with spam blog and SEO stuff"
Re: Training open-source LLMs on ChatGPT output is a really bad idea.
#8> So I can easily imagine a near future where the web will be flooded by LLM output or at least by content heavily inspired or edited by LLMs.
To be fair, we're already there, and we've been there for at least 10 years now. I'd wager >75% of the internet is garbage: auto-generated blog posts, programmatically-permuted ads, YouTube videos that mainly regurgitate other sources. Email is mostly garbage and the only reason it's usable is because spam filters have gotten pretty good. Even non-trivial amounts of heavily-curated social media (Twitter/FB/IG) is purely spam.
Re: Training open-source LLMs on ChatGPT output is a really bad idea.
#9I wonder how OpenAI are going to avoid the problem after the web is littered with its content?
They probably fingerprint their generated content.
Re: Training open-source LLMs on ChatGPT output is a really bad idea.
#10I wonder how OpenAI are going to avoid the problem after the web is littered with its content?
Presumably the ChatGPT content that makes it onto the web is at the very least curated by humans, making that text on average slightly higher quality than the raw output of ChatGPT. If that's the case than you would expect model performance to continue to improve even if the dataset is polluted.