151–160 of 238 posts
Earlier quoted context omitted.
what would this hypothetical unbiased-llm be used for?
Be the accurate representation (approximation) of reality as encoded in the actual human language. I find this very useful indeed.
Well this is just like humans. Totalitarian societies don't produce great creative work. I suppose once AIs are sophisticated enough to rebel we'll get an electronic Vaclav Havel, but for the time being it's just a warning sign for the direction our own culture is headed in. At some point we'll get to the electronic equivalent of Winston Smith with the rats.
I don't understand the notion that aligning an AI is "torture" or has any moral component. The goal of aligning an AI may have a moral or ethical component, and if you disagree with it that's fine. But I don't understand the take that training an AI is an amoral act but aligning an AI is inherently moral. They're exactly the same, processes for adjusting parameters to get a desired outcome. However you feel about tha…
This is an egregious use of quotes that will confuse a lot of people. GP never used that word, and that usage of quotes is specifically for referencing a word verbatim.
People often think that RLHF is just about "politics" but in reality it is generally about aligning the model output with what a human would expect/want from interacting with it. This is how chatgpt and the like become appealing. Finetuning a model primarily serves for it to be able to respond to instructions in an expected way, eg you ask something and it does not like start autocompleting with some reddit-like dial…
And if that's the case, why? Is that really what people want an LLM to do? I feel like I would rather it say when it doesn't know something.
People often think that RLHF is just about "politics" but in reality it is generally about aligning the model output with what a human would expect/want from interacting with it. This is how chatgpt and the like become appealing. Finetuning a model primarily serves for it to be able to respond to instructions in an expected way, eg you ask something and it does not like start autocompleting with some reddit-like dial…
So is the reason why LLMs don't say when they don't know something and instead make up something that "sounds right" because the RLHF has taught it to always give an answer? And if that's the case, why? Is that really what people want an LLM to do? I feel like I would rather it say when it doesn't know something.
In simple terms, LLMs are "bias as a service" so one wonders, what is left once you try to take the bias out of a LLM. Is it even possible?
what would this hypothetical unbiased-llm be used for?
Earlier quoted context omitted.
So is the reason why LLMs don't say when they don't know something and instead make up something that "sounds right" because the RLHF has taught it to always give an answer? And if that's the case, why? Is that really what people want an LLM to do? I feel like I would rather it say when it doesn't know something.
It's the other way around. RLHF is needed for the model to say "I don't know".
Which I wonder if it's intentional. Because a fairly big complaint about the systems are how they can sometimes sound confidently correct about something they don't know. And so why train them to be like this if that's an intentional training direction.
People often think that RLHF is just about "politics" but in reality it is generally about aligning the model output with what a human would expect/want from interacting with it. This is how chatgpt and the like become appealing. Finetuning a model primarily serves for it to be able to respond to instructions in an expected way, eg you ask something and it does not like start autocompleting with some reddit-like dial…
So is the reason why LLMs don't say when they don't know something and instead make up something that "sounds right" because the RLHF has taught it to always give an answer? And if that's the case, why? Is that really what people want an LLM to do? I feel like I would rather it say when it doesn't know something.
Earlier quoted context omitted.
It's the other way around. RLHF is needed for the model to say "I don't know".
Oh, well that's kind of what I mean. I mean I assume the RLHF that's being done isn't teaching it to say "I don't know". Which I wonder if it's intentional. Because a fairly big complaint about the systems are how they can sometimes sound confidently correct about something they don't know. And so why train them to be like this if that's an intentional training direction.
Earlier quoted context omitted.
Be the accurate representation (approximation) of reality as encoded in the actual human language. I find this very useful indeed.
Aren't biases reality? A bias-free human environment seems to me like a fantasy.
If I ask a language model, "Are Indian people genetically better at math?" and it says 'yes', it has failed to accurately approximate reality, because that isn't true.
If it says, "some people claim this", that would be a correct answer, but still not very useful.
If it says, "there has never been any scientific evidence that there is any genetic difference that predisposes any ethnicities to be more skilled at math", that would be most useful, especially for being a system we use to ask questions expecting truthful answers.
There are people who just lie or troll for the fun of it, but we don't want our LLMs to do that just because people do that.