71–80 of 173 posts
It's almost as if we trained LLMs on text produced by people.
My favorite is:
No one is as dumb as all of us.
* Pseudo Intelligence
At least LLMs are very often ready so acknowledge they might be wrong. It can be incredibly hard to get a person to acknowledge that they might be remotely wrong on a topic they really care about. Or, for some people, the thought that they might be wrong about anything attall is just like blasphemy to them.
"Acknowledging they might be wrong" makes them sound like more than token predictors trained on polite sounding text.
Is my understanding wrong that LLMs are trained to emulate observed human behavior in their training data? From that follows that LLMs fit to produce all kinds of human biases. Like preferring the first choice out of many, and the last our of many (primacy biases). Funnily the LLM might replicate the biases slightly wrong and by doing so produce new derived biases.
This understanding is incomplete in my opinion. LLMs are more than emulating observed behavior. In the pre-training phase tasks like masked language model indeed train the model to mimic what they read (which of course contains lots of bias); but in the RLHF phase, the model tries to generate the best response judged by human evaluations (who tries to eliminate as much bias as possible in the process). In other words…
How can the RLHF phase eliminate bias if it uses a process(human input) that has the same biases as the pre-training(human input)?
It seems like an incredibly bad outcome if we accept "AI" that's fundamentally flawed in a way similar to if not worse than humans and try to work around it rather than relegating it to unimportant tasks while we work towards a standard of intelligence we'd otherwise expect from a computer.
LLMs certainly appear to be the closest to real AI that we've gotten so far. But I think a lot of that is due to the human bias that language is a sign of intelligence and our measuring stick is unsuited to evaluate software specifically designed to mimic the human ability to string words together. We now have the unreliability of human language processes without most of the benefits that comes from actual human level intelligence. Managing that unreliability with systems designed for humans bakes in all the downsides without further pursuing the potential upsides from legitimate computer intelligence.
There has been some good research published on this topic of how RLHF, ie aligning to human preferences easily introduces mode collapse and bias into models. For example, with a prompt like: "Choose a random number", the base pretrained model can give relatively random answers, but after fine tuning to produce responses humans like, they become very biased towards responding with numbers like "7" or "42".
Human level artificial intelligence has never had much appeal to me, there are enough idiots in the world, why do we need artificial ones? Ie if average machine intelligence mirrored human IQ distribution?
But an LLM can't be held accountable.. neither can most employees, but we often forget that :)
Earlier quoted context omitted.
LLMs don't emulate human behavior. They spit out chunks of words in an order that parrots some of their training data.
Is this just pedantry or is there some insight to be gleaned by the distinction you made?
I believe the distinction they're trying to make is between "sounding like a human"(being able to create output that we understand as language) and "thinking like a human"(having the capacity for experience, empathy, semantic comprehension, etc.)
At least LLMs are very often ready so acknowledge they might be wrong. It can be incredibly hard to get a person to acknowledge that they might be remotely wrong on a topic they really care about. Or, for some people, the thought that they might be wrong about anything attall is just like blasphemy to them.
When you don't do that sufficiently you run the risk of producing the "Sydney" personality that Bing Chat had, which would argue back, and could go totally feral defending its incorrect beliefs about the world, to the point of insulting and belittling the user.