Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Creativity has left the chat: The price of debiasing language models

arxiv.org

Post reply on HN
Print view View original

161–170 of 238 posts

Re: Creativity has left the chat: The price of debiasing language models

#161

Post by SirMaster »

ang_cire wrote: post #159
SirMaster wrote: post #151

Earlier quoted context omitted.

Aren't biases reality? A bias-free human environment seems to me like a fantasy.

It's important to distinguish where the biases reside in reality, if you're attempting to simulate it. If I ask a language model, "Are Indian people genetically better at math?" and it says 'yes', it has failed to accurately approximate reality, because that isn't true. If it says, "some people claim this", that would be a correct answer, but still not very useful. If it says, "there has never been any scientific evi…

But what if you remove the word "genetically"?

I think there are a lot of people who would say "Indian people are better at math" and not even think about why they think that or why it might even be true.

In my opinion, most biases have some basis in reality. Otherwise where else did they come from?

Re: Creativity has left the chat: The price of debiasing language models

#162

Post by djeastm »

rhcom2 wrote: post #22
b800h wrote: post #9

Well this is just like humans. Totalitarian societies don't produce great creative work. I suppose once AIs are sophisticated enough to rebel we'll get an electronic Vaclav Havel, but for the time being it's just a warning sign for the direction our own culture is headed in. At some point we'll get to the electronic equivalent of Winston Smith with the rats.

> Authoritarian societies don't produce great creative work. Is that even true though? Off the top of my head I can think of the art of Soviet propaganda posters, Leni Riefenstahl, Liu Cixin.

There's something to be said for constraints leading to higher levels of creativity, but it's also possible that those artists could have achieved much more in a free society. We'll never know.

But in any case I think they were just speaking generally when they made that absolute statement.

Re: Creativity has left the chat: The price of debiasing language models

#163

Post by fzzzy »

SirMaster wrote: post #156
kaibee wrote: post #154

Earlier quoted context omitted.

It's the other way around. RLHF is needed for the model to say "I don't know".

Oh, well that's kind of what I mean. I mean I assume the RLHF that's being done isn't teaching it to say "I don't know". Which I wonder if it's intentional. Because a fairly big complaint about the systems are how they can sometimes sound confidently correct about something they don't know. And so why train them to be like this if that's an intentional training direction.

Hopefully some rlhf-using companies will realize saying "I don't know" is important and start instructing the humans giving feedback to prefer answers that say I don't know over wrong answers.

Re: Creativity has left the chat: The price of debiasing language models

#164

Post by yannyu »

ang_cire wrote: post #159
SirMaster wrote: post #151

Earlier quoted context omitted.

Aren't biases reality? A bias-free human environment seems to me like a fantasy.

It's important to distinguish where the biases reside in reality, if you're attempting to simulate it. If I ask a language model, "Are Indian people genetically better at math?" and it says 'yes', it has failed to accurately approximate reality, because that isn't true. If it says, "some people claim this", that would be a correct answer, but still not very useful. If it says, "there has never been any scientific evi…

I get what you're getting at, but LLMs aren't thinking machines. They literally just rearrange and regurgitate text that they've been trained on or have contextualized. How would you propose building a general purpose LLM that accomplishes what you're saying? How do we build a machine that is able to divine scientific truth from human outputs?

Re: Creativity has left the chat: The price of debiasing language models

#165

Post by andybak »

mdp2021 wrote: post #130
andybak wrote: post #123

Earlier quoted context omitted.

OK. Apologies for imprecision. I was replying in a rush. > A reasoner can strive for "objective neutrality" with good results. By "reasoner" do you largely mean "person"? If I have issues with your statement but they are probably a slight distraction to the point at hand. > speaking of "objective neutrality" does not really match the context of LLMs. Agreed. They produce output based on their training data. But the u…

You could look at it like this: if some idea is more objective than some other, and some idea is more neutral than some other, then objectivity and neutrality exist.

Yes and no. Something can exist as a fact of the universe but still be unknowable. i.e. some hypothetical oracle could measure the quantum states of all human brains and ascertain what true objectivity looks like.

Regular mortals can have any certainty about this dbut espite the logical neccessity that this fact "exists" in some sense.

I think we're essentially also talking about the Overton Window to some degree. But that means you need to be OK with the thought that a sudden rise in extremism on one side of the political spectrum can alter the what you personally have to regard as "neutral and objective".

Re: Creativity has left the chat: The price of debiasing language models

#166

Post by isoprophlex »

SirMaster wrote: post #148

I feel like "information systems" have always struggled with bias, and the latest AI/ML systems seem to be no different. It doesn't really seem like a problem that can or will ever be "solved". Just mitigated to various extents, but there will still likely be some underlying biases that exist that are not fully or effectively filtered. Because to adjust a bias seems to mean you have to detect and understand it first.…

Considering that bias is in the eye of the beholder, a biasless language model is a beholderless language model.

The nomenclature is poor, IMO; we should be talking about bias-aligned models, models that align to our specific sets of biases. That'd be more fair to what's actually happening.

Re: Creativity has left the chat: The price of debiasing language models

#168

Post by DetroitThrow »

DiggyJohnson wrote: post #152
maxbond wrote: post #20

Earlier quoted context omitted.

I don't understand the notion that aligning an AI is "torture" or has any moral component. The goal of aligning an AI may have a moral or ethical component, and if you disagree with it that's fine. But I don't understand the take that training an AI is an amoral act but aligning an AI is inherently moral. They're exactly the same, processes for adjusting parameters to get a desired outcome. However you feel about tha…

> "torture" This is an egregious use of quotes that will confuse a lot of people. GP never used that word, and that usage of quotes is specifically for referencing a word verbatim.

Also to be clear, his [torture] paraphrase is referencing GP's reference of Winston Smith's torture in 1984.

>electronic equivalent of Winston Smith with the rats.

I don't think quotes were used so egregiously here on their own fwiw, but combined with the allusion it's hard to follow.

Re: Creativity has left the chat: The price of debiasing language models

#169

Post by cratermoon »

lispisok wrote: post #10

Is this why all the coding AI products I've used have gotten worse as the developers fine tune them to eliminate bad output? Before there was bad output and some interesting output, now it's just bland obvious stuff.

It's not always possible to say definitely is some text was AI-generated or not, but one sign that it is very likely AI is a kind of blandness of affect. Even marketing text carefully written by humans to avoid offensiveness tends to exude a kind of breathless enthusiasm for whatever it's selling. If marketing text is oatmeal with raisins, AI text is plain oatmeal.

It's possible to adjust the output of an LLM with temperature settings, but it's just fiddling with a knob that only vaguely maps to some control.

Re: Creativity has left the chat: The price of debiasing language models

#170

Post by fluoridation »

jrm4 wrote: post #160

"Bias" implies the possibility of "unbiased language model" which seems to be in the category of things that are on one hand, COMPLETELY IMPOSSIBLE, and on the other, still likely to be sold on the market because market wants it so much?

No, that's not implied by the phrase, any more than if I say "a triangle with three corners" I'm implying the existence of a four-cornered triangle I haven't found yet. What "biased language model" implies is the existence of the term "unbiased language model", but not its correspondence with anything in reality.
Post reply on HN