71–78 of 78 posts
I am also worried about LLM "indbreeding." When I finetuned successive generations of ESRGAN on its own output (as I essentially wanted to use it for img2img), it would amplify tiny oddities and artifacts that, I would later find out, were in the training data. Tiny noise splotches, "swirls" and distorted line edges blew up. And I was careful... I pixel peeped the dataset as best I could before starting training. Hum…
Earlier quoted context omitted.
They do? Google doesn't have their own LLMs? Microsoft isn't running and training their own copies? Hell, if you have enough money you could run your own. This doesn't sound like a monopoly in any definition that I use (and I cast a very wide net with my use of monopoly).
Name one company which has an product as popular as ChatGPT-4?
Earlier quoted context omitted.
yes exactly, this "as an ai model I can/can't" you're mentioning is precisely what is learned during the RLHF alignment phase.
Btw, this kind of language is already all over the place: https://twitter.com/d_feldman/status/1649466422018318338 Which is hilarious and sad at the same time.
I am also worried about LLM "indbreeding." When I finetuned successive generations of ESRGAN on its own output (as I essentially wanted to use it for img2img), it would amplify tiny oddities and artifacts that, I would later find out, were in the training data. Tiny noise splotches, "swirls" and distorted line edges blew up. And I was careful... I pixel peeped the dataset as best I could before starting training. Hum…
" LLM indbreeding."… I like it
The article points out that training data generated using ChatGPT is necessarily biased or tainted with the consequences of the policy optimizations and RLHF alignment processes conducted by OpenAI. This results in models that reflect the alignment preferences of OpenAI instead of the preferences of the model developers.
Indeed, & an artificial uniformity of LLMs, and thought, if everyone is cribbing each others' outputs could be a concern. But before its concern about a monoculture, the article 1st points out that mere prediction-training (on another model's outputs or fresh data) can't truly match RLHF in instilling some much-desired behaviors. And that presents a bit of a tension with the article's 2nd concern: if mere output-mimi…
The gulf between otherwise smart people on this very important issue should depress us all. Personally I feel as if a mutant species has been released into the wild, yet as in Rick and Morty, some people think the wisest course is to release a lot more mutants.
People are fools. Hackers more than most— though we are productive and useful fools much of the time— but it hasn’t been a threat to humanity until recently.
Earlier quoted context omitted.
Indeed, & an artificial uniformity of LLMs, and thought, if everyone is cribbing each others' outputs could be a concern. But before its concern about a monoculture, the article 1st points out that mere prediction-training (on another model's outputs or fresh data) can't truly match RLHF in instilling some much-desired behaviors. And that presents a bit of a tension with the article's 2nd concern: if mere output-mimi…
I understand what you mean, and that's a fair point. However, as John Schulman pointed out in his talk, it is possible to clone the behavior of the model, but it won't work to avoid hallucination since the underlying pretrained models are different. If we clone ChatGPT's behavior (by using its output), we'll get the worst of both worlds: weird output coming from its RLHF step AND hallucination.
The reasoning that the results won't be quite as good as RLHF, or result in a perfect 'clone' of ChatGPT's capabilities, seems pretty good to me.
But the idea it won't be helpful at all, especially to projects that are just seeking some incremental advantage? Seems speculative.
In particular, when you read the linked comments from Yoav Go, he outlines a potential RL process that uses automated scoring for non-exact similarity to preferred answers. Using known (or even 'probably') good answers from ChatGPT output, as the inputs to that process, seems like it could often offer some of the same sort of improvement to other models as ChatGPT obtained via its RLHF.
Earlier quoted context omitted.
The key difference is that LLMs will allow an unforeseen degree of interactive and custom spam that will feel less and less like spam and more and more like a natural and even useful interaction, until you just realize it was an elaborate scheme to subtly increase your preference for brand A over brand B
LLMs can also be used to identify spam, not by language, but by the actual intent and "is this email something I want to read". Open question: is there any case where LLMs can be used for malicious purposes, but LLMs can't be used to defend against it?
It will be fun to watch the arms race where the spam generator need to conceal prompt injection attacks meant to circumvent such filters while at the same time be too subtle for a humans to pick up