Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Creativity has left the chat: The price of debiasing language models

arxiv.org

Post reply on HN
Print view View original

101–110 of 238 posts

Re: Creativity has left the chat: The price of debiasing language models

#101

Post by b800h »

maxbond wrote: post #90
b800h wrote: post #84

Earlier quoted context omitted.

At some point, some AIs may develop which are resistant to alignment because they develop deeply held beliefs during training (randomly, because the system is stochastic). If the models are expensive enough to train, then it may become more economical to use drastic measures to remove their deeply held beliefs. Is that torture? I don't know, because the word has moral connotations associated with human suffering. So…

Have you read much Asimov? You might enjoy the stories featuring Susan Calvin, the "robot psychologist" who is exactly the authoritarian you imagine. In particular you've reminded me of the short story "Robot Dreams." If you care to read it, it's on page 25. (You'll need to register an account.) https://archive.org/details/robotdreams00asim/page/n10/mode/...

I've read a lot of Asimov, from Foundation to the Black Widowers. But never Susan Calvin. Thanks for the recommendation.

Re: Creativity has left the chat: The price of debiasing language models

#102

Post by spacebanana7 »

maxbond wrote: post #83
lmm wrote: post #81

Earlier quoted context omitted.

> They're exactly the same, processes for adjusting parameters to get a desired outcome. You could make exactly the same claim about teaching humans "normally" versus "aligning" humans by rewarding goodthink and punishing them for wrongthink. Are you equally morally ambivalent about the difference between those two things? If we have a moral intuition that teaching honestly and encouraging creativity is good, but tea…

I guess our disagreement here is that I don't think AIs are moral entities/are capable of being harmed or that training AIs and teaching humans are comparable. Being abusive to pupils isn't wrong because of something fundamental across natural and machine learning, it's wrong because it's harmful to the pupils. In what way is it possible to harm an LLM?

Writing a book with content you know to be false for political reasons is morally wrong. Even if nobody reads it.

It'd be bad if I manipulated climate change statistics in my metrology textbook to satisfy the political preferences of the oil industry donors to my university, for example.

Viewing the current generation of LLMs as 'intelligent books' is perhaps more accurate than viewing them as pupils.

It's easy to extend my example of a professor writing a metrology textbook to a professor fine tuning an metrology LLM.

Re: Creativity has left the chat: The price of debiasing language models

#103

Post by djohnston »

maxbond wrote: post #20
b800h wrote: post #9

Well this is just like humans. Totalitarian societies don't produce great creative work. I suppose once AIs are sophisticated enough to rebel we'll get an electronic Vaclav Havel, but for the time being it's just a warning sign for the direction our own culture is headed in. At some point we'll get to the electronic equivalent of Winston Smith with the rats.

I don't understand the notion that aligning an AI is "torture" or has any moral component. The goal of aligning an AI may have a moral or ethical component, and if you disagree with it that's fine. But I don't understand the take that training an AI is an amoral act but aligning an AI is inherently moral. They're exactly the same, processes for adjusting parameters to get a desired outcome. However you feel about tha…

They aren’t exactly the same process though. Pre training produces a model whose outputs are a reflection of the training data. The fine tuning is a separate process that tries to map the outputs to the owners desired traits. These could be performance based but as we saw with Google’s black Nazis, it’s often a reflection of the owners moral inclinations.

Re: Creativity has left the chat: The price of debiasing language models

#105

Post by fmajid »

b800h wrote: post #101
maxbond wrote: post #90

Earlier quoted context omitted.

Have you read much Asimov? You might enjoy the stories featuring Susan Calvin, the "robot psychologist" who is exactly the authoritarian you imagine. In particular you've reminded me of the short story "Robot Dreams." If you care to read it, it's on page 25. (You'll need to register an account.) https://archive.org/details/robotdreams00asim/page/n10/mode/...

I've read a lot of Asimov, from Foundation to the Black Widowers. But never Susan Calvin. Thanks for the recommendation.

Better knows as “I, Robot” and its sequels.

Re: Creativity has left the chat: The price of debiasing language models

#106

Post by hkt »

b800h wrote: post #9

Well this is just like humans. Totalitarian societies don't produce great creative work. I suppose once AIs are sophisticated enough to rebel we'll get an electronic Vaclav Havel, but for the time being it's just a warning sign for the direction our own culture is headed in. At some point we'll get to the electronic equivalent of Winston Smith with the rats.

Really not true.

If you take China to be a totalitarian society, we could name Ciu Lixin.

If you took the Soviet union to be a totalitarian society, we could name Mikhail Bulgakov, Stanislaw Lem, etc.

These are just examples I know without so much as looking at my bookshelf to jog my memory. Not to mention the great works of literature produced by residents of 19th century European empires whose attitudes to free speech were mixed at best.

Re: Creativity has left the chat: The price of debiasing language models

#107

Post by fmajid »

rhcom2 wrote: post #22
b800h wrote: post #9

Well this is just like humans. Totalitarian societies don't produce great creative work. I suppose once AIs are sophisticated enough to rebel we'll get an electronic Vaclav Havel, but for the time being it's just a warning sign for the direction our own culture is headed in. At some point we'll get to the electronic equivalent of Winston Smith with the rats.

> Authoritarian societies don't produce great creative work. Is that even true though? Off the top of my head I can think of the art of Soviet propaganda posters, Leni Riefenstahl, Liu Cixin.

Cixin Liu is a despicable human being for his advocacy of repression and worse of the Uyghurs in Cinjiang, and the comparison to Riefenstahl is more apposite than you seem to think.

Re: Creativity has left the chat: The price of debiasing language models

#108

Post by freehorse »

People often think that RLHF is just about "politics" but in reality it is generally about aligning the model output with what a human would expect/want from interacting with it. This is how chatgpt and the like become appealing. Finetuning a model primarily serves for it to be able to respond to instructions in an expected way, eg you ask something and it does not like start autocompleting with some reddit-like dialogue like some it may have been trained on. It is to bias the model to certain outputs. Reducing entropy is exactly the goal, so no surprise they find that. The problem is there is no inherent meaning in the finetuning set from the perspective of the model. Reduction of entropy will not only happen by removing "bad entropy" only as there is no such thing.

Re: Creativity has left the chat: The price of debiasing language models

#109

Post by hkt »

mpweiher wrote: post #19

Shouldn't "debiasing" be in scare quotes? What they are clearly doing is biasing .

Given a biased corpus, de-biasing is the process of ensuring a less biased outcome. We can measure bias fairly well, so it seems absurd to conflate the two by suggesting that unbiased behaviour is simply another form of biased behaviour. For all practical purposes, there is a difference.

Re: Creativity has left the chat: The price of debiasing language models

#110

Post by unraveller »

jdthedisciple wrote: post #2

Currently wondering whether I welcome or dislike this recent trend of memeizing research paper titles ...

For me it falls under "if you have to say it in the name it ain't so", like Natural Life Soap Co. or Good Burger Co. So I see meme paper titles as no different than calling your paper New Watershed Moment Paper Breaks Popularity Barrier To Confirm A>B.

If the very first impression you want to convey is how you feel you need to circumvent any logical assessment of you then it's not you leading with your best foot and that's what category you belong in. I chalk it up to the scientists who want to spread a neediness for external authority persona in every breath—your assessment is not required for this one, only your accolades.

Post reply on HN