181–190 of 238 posts
How hard would it be to create a "raw" model on a corpus like Hacker News or Wikipedia? With "raw", I mean that it is simply trained to predict the next token and nothing else. Would be fun to play with such a model.
Obviously clustering is a different problem from inference, but they are all high dimensional vector spaces - it should be easy enough to take a fair clustering algorithm and modify it to generate continuous mappings instead of discrete groups. But if it all works, the LLM should be e.g. race-blind in that asking for a description of a rich man will give skin tones following population statistics but he will always be wearing an expensive suit. The question of what to protect is tricky though, e.g. age is often considered protected but if you ask for an old man with gray hair it would be surprising to get a retired age 30 person. So there is some subjectivity in designing the protected features dataset to show what should be considered similar or same-clusters.
But really the purpose of RLHF is to reduce toxicity. It should be possible to orthogonalize toxicity like everything else, then there would not be a reduction in generated races like the paper observed.
Earlier quoted context omitted.
> "torture" This is an egregious use of quotes that will confuse a lot of people. GP never used that word, and that usage of quotes is specifically for referencing a word verbatim.
Also to be clear, his [torture] paraphrase is referencing GP's reference of Winston Smith's torture in 1984. >electronic equivalent of Winston Smith with the rats. I don't think quotes were used so egregiously here on their own fwiw, but combined with the allusion it's hard to follow.
How hard would it be to create a "raw" model on a corpus like Hacker News or Wikipedia? With "raw", I mean that it is simply trained to predict the next token and nothing else. Would be fun to play with such a model.
There are some that exist. The problem is you need at least some RLHF to make it follow instructions instead of just predicting sentences.
Earlier quoted context omitted.
No, diversity isn't creativity. For example, we could search google for "great art" and if it produced a sample of one art work from ever decade of the last 500 years that would likely be highly diverse in style and content. If it returned a list of the best work from western Europe in the of the 18th century it would be rather consistent. Both lists would have the same amount of creativity though - 0.
"one art work from every decade of the last 500 years that would likely be highly diverse in style and content" It still might not be especially diverse if all 50 examples were from western European art. 500 years only takes us back to 1524 - not especially long and mostly from the same early modern period starting with the fall of Constantinople, the end of the Crusades, and the start of the Renaissance. I wouldn't…
Something I notice about text written by LLMs is how painfully obvious they are to identify sometimes. Recently I was watching a very well researched two hour video on Tetris World Records [1], but the sheer amount of text clearly "enhanced" by an LLM really made me uncomfortable. ChatGPT speaks a very specific, novel, dialect of English, which I've come to deeply despise. I'd always guessed it was caused by some kin…
Is this the first Summoning Salt video you've seen? I don't know enough to say that he doesn't use an LLM during his writing process, but I do know that I haven't noticed any appreciable difference between his newer videos and ones that were released before ChatGPT was made available. Is it possible that this is just the way he chooses to write his scripts that you interpret as sounding like they are written by an LL…
People often think that RLHF is just about "politics" but in reality it is generally about aligning the model output with what a human would expect/want from interacting with it. This is how chatgpt and the like become appealing. Finetuning a model primarily serves for it to be able to respond to instructions in an expected way, eg you ask something and it does not like start autocompleting with some reddit-like dial…
So is the reason why LLMs don't say when they don't know something and instead make up something that "sounds right" because the RLHF has taught it to always give an answer? And if that's the case, why? Is that really what people want an LLM to do? I feel like I would rather it say when it doesn't know something.
How hard would it be to create a "raw" model on a corpus like Hacker News or Wikipedia? With "raw", I mean that it is simply trained to predict the next token and nothing else. Would be fun to play with such a model.
"Bias" implies the possibility of "unbiased language model" which seems to be in the category of things that are on one hand, COMPLETELY IMPOSSIBLE, and on the other, still likely to be sold on the market because market wants it so much?
No, that's not implied by the phrase, any more than if I say "a triangle with three corners" I'm implying the existence of a four-cornered triangle I haven't found yet. What "biased language model" implies is the existence of the term "unbiased language model", but not its correspondence with anything in reality.
I had an argument with some people over what debiasing means. There is some interesting research on fair clustering that I think points the way. The way fair clustering works is that you take data with both protected and unprotected attributes, and then you orthogonalize the unprotected attributes based on the protected attributes. So for example, if race is protected and income is unprotected, but there is a strong…
The idea that the good data is secretly encoded in uncorrupted form within the bad data I think is a bad idea. It reminds me of trying to make bad mortgages into good CDOs.
> But really the purpose of RLHF is to reduce toxicity.
I don't think that's the goal, I think it's some people's goal. Those people have defined what "toxicity" means to them, and they're mistaking it for a universal. It's just a metaphor about poison, because poison is bad. It's not a coherent concept. For a business, it should be anything that drives customers away and affects profit. That can only be considered statistically: if some people think something is toxic, and other people think that not mentioning that thing is toxic, the winner is whoever improves the bottom line more or damages it less.
That's how the raw data ended up like it is in the first place.