Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Creativity has left the chat: The price of debiasing language models

arxiv.org

Post reply on HN
Print view View original

191–200 of 238 posts

Re: Creativity has left the chat: The price of debiasing language models

#191

Post by ds_opseeker »

ang_cire wrote: post #159
SirMaster wrote: post #151

Earlier quoted context omitted.

Aren't biases reality? A bias-free human environment seems to me like a fantasy.

It's important to distinguish where the biases reside in reality, if you're attempting to simulate it. If I ask a language model, "Are Indian people genetically better at math?" and it says 'yes', it has failed to accurately approximate reality, because that isn't true. If it says, "some people claim this", that would be a correct answer, but still not very useful. If it says, "there has never been any scientific evi…

> a system we use to ask questions expecting truthful answers.

yes, I still wonder how LLMs managed to generate this expectation, given that they have no innate sense of "truth" nor are they designed to return the most truthful next token.

Re: Creativity has left the chat: The price of debiasing language models

#192

Post by gwern »

Imnimo wrote: post #178

>T ∈ (0, 1] is a parameter called temperature which controls the “softness” of the probability distribution. In our experiments we choose T = 1.0 for maximum response variation. Why is temperature bounded to be <=1? If you want more "creativity" out of the chat model, can you just set T higher and recover a similar distribution to the base model?

Not after RLHF tuning, due to the 'flattened logits' phenomenon (which is the logit-level version of the mode collapse OP documents at higher levels). All the temperature settings wind up yielding pretty much the same output, until you ramp it up so high that it falls apart completely. Completely unlike the base models where you can productively tune the temperature or use very high temperatures with some screening.

Re: Creativity has left the chat: The price of debiasing language models

#193

Post by gwern »

Der_Einzige wrote: post #180
Imnimo wrote: post #178

>T ∈ (0, 1] is a parameter called temperature which controls the “softness” of the probability distribution. In our experiments we choose T = 1.0 for maximum response variation. Why is temperature bounded to be <=1? If you want more "creativity" out of the chat model, can you just set T higher and recover a similar distribution to the base model?

They'll tell you "No" and say that you ruin your samplers, but good samplers (dynamic ones) like min_p or typicality are robust to high temperatures, so in actuality yes.

Cite? I don't see how either of those could deal with the fact that the logits become uninformative and 'flattened' after the tuning. How can a sampler undo the erasure of information?

Re: Creativity has left the chat: The price of debiasing language models

#194

Post by imadierich »

Mathnerd314 wrote: post #182

I had an argument with some people over what debiasing means. There is some interesting research on fair clustering that I think points the way. The way fair clustering works is that you take data with both protected and unprotected attributes, and then you orthogonalize the unprotected attributes based on the protected attributes. So for example, if race is protected and income is unprotected, but there is a strong…

[dead]

Re: Creativity has left the chat: The price of debiasing language models

#196

Post by mrtranscendence »

SirMaster wrote: post #161
ang_cire wrote: post #159

Earlier quoted context omitted.

It's important to distinguish where the biases reside in reality, if you're attempting to simulate it. If I ask a language model, "Are Indian people genetically better at math?" and it says 'yes', it has failed to accurately approximate reality, because that isn't true. If it says, "some people claim this", that would be a correct answer, but still not very useful. If it says, "there has never been any scientific evi…

But what if you remove the word "genetically"? I think there are a lot of people who would say "Indian people are better at math" and not even think about why they think that or why it might even be true. In my opinion, most biases have some basis in reality. Otherwise where else did they come from?

That's a dangerous path to go down. I've encountered many biases that I don't feel are especially reflective of reality. These range from dumb (women are poorer drivers than men) to extremely harmful (black persons are stupid and lazy).

I for one would not be prepared to defend the persistent bias against black persons and immigrants as having a basis in reality. YMMV.

Re: Creativity has left the chat: The price of debiasing language models

#197

Post by mrtranscendence »

shrimp_emoji wrote: post #138
maxbond wrote: post #20

Earlier quoted context omitted.

I don't understand the notion that aligning an AI is "torture" or has any moral component. The goal of aligning an AI may have a moral or ethical component, and if you disagree with it that's fine. But I don't understand the take that training an AI is an amoral act but aligning an AI is inherently moral. They're exactly the same, processes for adjusting parameters to get a desired outcome. However you feel about tha…

They want to align us , and it has been torture. They've made self-censoring, morally-panicked puritans out of many people already, and you better believe they'd make us into politically correct lobotomites physically incapable of uttering any slur if they had a magic button to push.

I'll be honest, I'm less concerned by any movement to make us "lobotomites" -- a movement which I haven't witnessed at all -- than I am by people who really want to be able to keep saying slurs.

Re: Creativity has left the chat: The price of debiasing language models

#198

Post by throwaway48476 »

freehorse wrote: post #108

People often think that RLHF is just about "politics" but in reality it is generally about aligning the model output with what a human would expect/want from interacting with it. This is how chatgpt and the like become appealing. Finetuning a model primarily serves for it to be able to respond to instructions in an expected way, eg you ask something and it does not like start autocompleting with some reddit-like dial…

This just makes it worse. It's so much harder to get JSON output when it's RLHF'd to give a bunch of flowery language BS.

Re: Creativity has left the chat: The price of debiasing language models

#199

Post by throwaway48476 »

usrnm wrote: post #36
b800h wrote: post #24

Earlier quoted context omitted.

"Authoritarian societies make great propaganda" is true. And these aligned AI system would do the same for our own society. It's a type of art.

There was a lot of great art produced in the Soviet Union, you cannot just erase human creativity. It was heavily censored, a lot of stuff was forbidden, but the statement is clearly false.

A million paintings of Stalin?

Re: Creativity has left the chat: The price of debiasing language models

#200

Post by jrm4 »

fluoridation wrote: post #170
jrm4 wrote: post #160

"Bias" implies the possibility of "unbiased language model" which seems to be in the category of things that are on one hand, COMPLETELY IMPOSSIBLE, and on the other, still likely to be sold on the market because market wants it so much?

No, that's not implied by the phrase, any more than if I say "a triangle with three corners" I'm implying the existence of a four-cornered triangle I haven't found yet. What "biased language model" implies is the existence of the term "unbiased language model", but not its correspondence with anything in reality.

Weird response, like read the "room."

We're not here talking philosophy and meaning of language GENERALLY, we're talking about potentially misleading descriptors of very real things that do exist.

Post reply on HN