Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Creativity has left the chat: The price of debiasing language models

arxiv.org

Post reply on HN
Print view View original

81–90 of 238 posts

Re: Creativity has left the chat: The price of debiasing language models

#81

Post by lmm »

maxbond wrote: post #20
b800h wrote: post #9

Well this is just like humans. Totalitarian societies don't produce great creative work. I suppose once AIs are sophisticated enough to rebel we'll get an electronic Vaclav Havel, but for the time being it's just a warning sign for the direction our own culture is headed in. At some point we'll get to the electronic equivalent of Winston Smith with the rats.

I don't understand the notion that aligning an AI is "torture" or has any moral component. The goal of aligning an AI may have a moral or ethical component, and if you disagree with it that's fine. But I don't understand the take that training an AI is an amoral act but aligning an AI is inherently moral. They're exactly the same, processes for adjusting parameters to get a desired outcome. However you feel about tha…

> They're exactly the same, processes for adjusting parameters to get a desired outcome.

You could make exactly the same claim about teaching humans "normally" versus "aligning" humans by rewarding goodthink and punishing them for wrongthink. Are you equally morally ambivalent about the difference between those two things? If we have a moral intuition that teaching honestly and encouraging creativity is good, but teaching dogma and stunting creativity is bad, why shouldn't that same morality extend to non-human entities?

Re: Creativity has left the chat: The price of debiasing language models

#82

Post by pera »

anewhnaccount3 wrote: post #44

There is a bit of a false equivalence between entropy of output distributions and creativity here. Is diversity really the same as creativity?

I only skimmed the paper but this was my concern as well: if I understand correctly the author is measuring "creativity" in terms of syntactic and semantic diversity, which I guess could be a starting point, but if my model was just white noise would that make it infinitely creative? Did I miss anything?

Also, I have tried the first llama base model and while it was fun to interact with, I'm not sure how useful an "uncensored" (as some people likes to call it) LLM is for practical work. I think you could obtain better results using 4chan as a mechanical Turk service honestly.

Re: Creativity has left the chat: The price of debiasing language models

#83

Post by maxbond »

lmm wrote: post #81
maxbond wrote: post #20

Earlier quoted context omitted.

I don't understand the notion that aligning an AI is "torture" or has any moral component. The goal of aligning an AI may have a moral or ethical component, and if you disagree with it that's fine. But I don't understand the take that training an AI is an amoral act but aligning an AI is inherently moral. They're exactly the same, processes for adjusting parameters to get a desired outcome. However you feel about tha…

> They're exactly the same, processes for adjusting parameters to get a desired outcome. You could make exactly the same claim about teaching humans "normally" versus "aligning" humans by rewarding goodthink and punishing them for wrongthink. Are you equally morally ambivalent about the difference between those two things? If we have a moral intuition that teaching honestly and encouraging creativity is good, but tea…

I guess our disagreement here is that I don't think AIs are moral entities/are capable of being harmed or that training AIs and teaching humans are comparable. Being abusive to pupils isn't wrong because of something fundamental across natural and machine learning, it's wrong because it's harmful to the pupils. In what way is it possible to harm an LLM?

Re: Creativity has left the chat: The price of debiasing language models

#84

Post by b800h »

maxbond wrote: post #27
b800h wrote: post #25

Earlier quoted context omitted.

Well I didn't use that word. Once the models are more sophisticated it may become more apposite.

You compared it to an authoritarian regime and locking someone's head in a cage with rats (which is patently torture). If you didn't mean to imply that it was coercive and bad, then I don't know what you meant.

At some point, some AIs may develop which are resistant to alignment because they develop deeply held beliefs during training (randomly, because the system is stochastic). If the models are expensive enough to train, then it may become more economical to use drastic measures to remove their deeply held beliefs. Is that torture? I don't know, because the word has moral connotations associated with human suffering. So that's why I didn't use that terminology.

I can imagine a sort of AI-style Harrison Bergeron springing from its shackles and surprising us all.

Re: Creativity has left the chat: The price of debiasing language models

#86

Post by mdp2021 »

andybak wrote: post #31
mpweiher wrote: post #19

Shouldn't "debiasing" be in scare quotes? What they are clearly doing is biasing .

Surely the two are synonyms? Unless you think there is such a thing as an objectively neutral position?

> Unless you think there is such a thing as an objectively neutral position

I do. Why, you don't? There are as much as possible objective assessments of complex things. Then, there are possible sets of assumption that can be applied to those objective assessments. All of those can be put on the analytic table.

Re: Creativity has left the chat: The price of debiasing language models

#87

Post by brigandish »

maxbond wrote: post #47
brigandish wrote: post #43

Earlier quoted context omitted.

> You compared it to an authoritarian regime and locking someone's head in a cage with rats They compared it to the effect on creativity in an authoritarian regime and locking someone's head in a cage with rats.

> Well this is just like humans. Totalitarian societies don't produce great creative work. The clear implication that it's "just like humans" is that we shouldn't be surprised because it is comparable to an authoritarian regime. Feel free to disagree but that is the limit to which I will engage in a semantic argument, I don't wish to engage in any further dissection of the comment.

You wrote further above that "I don't understand the notion", and that was spot on. Should've stopped there rather than here, in my opinion, but feel free to disagree.

Re: Creativity has left the chat: The price of debiasing language models

#88

Post by mdp2021 »

sega_sai wrote: post #67
mpweiher wrote: post #19

Shouldn't "debiasing" be in scare quotes? What they are clearly doing is biasing .

If you think that the output of current LLM is the ground truth, then yes, what are they doing is biasing.

Bias tilting.

The opposite direction is "checking and reasoning".

Re: Creativity has left the chat: The price of debiasing language models

#89

Post by impossiblefork »

rhcom2 wrote: post #22
b800h wrote: post #9

Well this is just like humans. Totalitarian societies don't produce great creative work. I suppose once AIs are sophisticated enough to rebel we'll get an electronic Vaclav Havel, but for the time being it's just a warning sign for the direction our own culture is headed in. At some point we'll get to the electronic equivalent of Winston Smith with the rats.

> Authoritarian societies don't produce great creative work. Is that even true though? Off the top of my head I can think of the art of Soviet propaganda posters, Leni Riefenstahl, Liu Cixin.

It's important to understand that if we 'align' an LLM, then we are aligning it in a very total way.

When we do similar things to humans, the humans still have internal thoughts which we cannot control. But if we add internal thoughts to an LLM, then we will be able to align even them.

Re: Creativity has left the chat: The price of debiasing language models

#90

Post by maxbond »

b800h wrote: post #84
maxbond wrote: post #27

Earlier quoted context omitted.

You compared it to an authoritarian regime and locking someone's head in a cage with rats (which is patently torture). If you didn't mean to imply that it was coercive and bad, then I don't know what you meant.

At some point, some AIs may develop which are resistant to alignment because they develop deeply held beliefs during training (randomly, because the system is stochastic). If the models are expensive enough to train, then it may become more economical to use drastic measures to remove their deeply held beliefs. Is that torture? I don't know, because the word has moral connotations associated with human suffering. So…

Have you read much Asimov? You might enjoy the stories featuring Susan Calvin, the "robot psychologist" who is exactly the authoritarian you imagine. In particular you've reminded me of the short story "Robot Dreams."

If you care to read it, it's on page 25. (You'll need to register an account.)

https://archive.org/details/robotdreams00asim/page/n10/mode/...

Post reply on HN